AI that runs right on your phone or laptop instead of on a faraway server.
On-device AI means the model runs on your own device, such as a phone or laptop. Your request doesn't have to travel to a distant data center, so it can work offline and often answers faster. What you type or photograph also doesn't have to leave the device.
The catch is that phones have limited memory and battery, so big models don't fit. Instead they use small models, like the roughly 3-billion-parameter model Apple described in 2024, often shrunk further with tricks such as quantization. Many phones split the work: easy jobs stay on the device and hard ones go to a server.
Think of it like this
It's the difference between walking to the library every time you have a question and flipping open a pocket dictionary from your bag. The dictionary is instant, and nobody else sees what you looked up. For the really hard stuff, though, you still need the library.
For example
If your phone can still take dictation or read the text in a photo in airplane mode, that's on-device AI at work.
Records about this
Source-checked moments from AI history.
A milestone at which small, on-device-class models began to combine multimodality with competitive benchmark scores
The first large-scale case of OS-level on-device LLM integration