Somewhere between "your phone now uploads everything to the cloud" and "70 billion parameters, fully offline, on your phone" lies the truth. And honestly, it's still impressive.
What's actually running on the chip
Apple's current on-device model for Apple Intelligence has roughly 3 billion parameters — much smaller than ChatGPT or Claude, but built specifically for what the Neural Engine (the chip's dedicated AI processing unit) does fastest. Result: response times under 200 milliseconds, directly on the device, with no internet connection at all.
Qualcomm takes a slightly different approach. Its current Hexagon NPU (the AI unit inside the Snapdragon chip) is, according to the manufacturer, about 37% faster than the previous generation and supports a new, highly efficient precision format called INT2 (a way to compress models harder without making them useless). In demos, that was enough to run larger language models directly on the phone — at over 200 tokens per second (a token is roughly one syllable) with a context window of up to 128,000 tokens, or several hundred pages of text.
That's not "GPT-4 in your pocket." But it's not nothing, either.
Why this actually matters for privacy
The real point here isn't raw compute — it's where your data stays. When processing happens locally on the chip, your photos, voice notes, or emails never have to leave the device's storage. No upload, no server that could read along — at least not for that particular request.
That's genuine progress compared to the default logic of the last few years, where practically every AI request automatically ended up in some cloud. But watch out for the next round of hype: "on-device" doesn't automatically mean "everything stays local." Most manufacturers run a hybrid setup — simple requests stay on the chip, more complex ones still get routed to bigger models in the cloud (Apple, for example, uses "Private Cloud Compute," a server setup with its own privacy promises — but a server nonetheless). Which request goes where is decided by the software, not by you, at least not by default.
There's a catch that often gets lost here, too: "on-device" only covers the AI processing itself — not the rest of your phone's privacy picture. iCloud Backup, Google Photos backup, or WhatsApp cloud backups run in the background by default for most people and upload the exact same photos, messages, and emails to the cloud — completely independent of whether an AI feature touched them locally. The Neural Engine's privacy win only applies to that one AI request, not to your data overall. If your backup is already switched on, that photo was probably sitting in the cloud long before any AI ever looked at it.
What you can actually do right now
It's worth checking your settings. Current iPhones and Android flagships increasingly show whether an AI feature runs locally or in the cloud — usually somewhere under "Apple Intelligence" or the manufacturer's privacy or AI settings. If you're particularly cautious, you can often disable cloud features on purpose and stick to on-device processing only — at the cost of slightly weaker answers for more complicated tasks.
One more thing: AI settings and backup settings are two separate battles. It's worth checking iCloud Backup, or your Google Photos/Google One backup, on their own — that's what actually decides whether your photos and messages end up in a cloud at all, independent of any AI feature.
Is this relevant for you already?
If you own a current flagship phone: yes, directly. These features already run in the background without you having to flip a switch. If your phone is older: the NPU power needed for bigger local models is still fairly new — older chips are limited to simpler tasks like text recognition or photo categorization.
Either way, the trend points somewhere pleasant: AI that doesn't phone the internet for every little thing. Not a reason to declare privacy solved across the board — the AI request itself stays local, but your backup habits decide whether the rest of your data does too. Still, it's a solid step where privacy and technical progress happen to point in the same direction, for once.