Did your voice ever leave the machine? A checklist for on-device AI
"Runs locally" is everywhere now, and it is routinely used to mean four different things. You ask whether what you said will be sent anywhere; the answer you get may be that the model weights live on your device. Both statements can be true and still not answer your question.
Here is a checklist you can actually use, followed by what it looks like when applied to one sentence.
Four words that are not synonyms
📖 Glossary
AI Box (also known as Agent Computer / Agent PC), is a dedicated local hardware device that runs AI Agents. Pre-installed with an AI agent management system, plug-and-play, running 24/7. Users can remotely command AI to work via Discord, Slack, Telegram, WhatsApp, and more.
On-device: the computation happens on this device; inference does not borrow a server. Offline: the whole action completes with no network and no outside request. Local storage: the data is written to this device, not someone else's database. Private: you set the rules for who can read it, who can delete it and how long it stays.
Any one of these can hold without the others. A model can run on-device while every session still checks an account online. A file can sit on local storage while a sync feature quietly backs it up to the cloud. A feature can work offline even though the model it needed on first use came from outside. Separate the four and a lot of marketing copy stops being ambiguous.
Five questions that locate the answer

Whatever the vendor calls it, ask these five.
- Does the whole action still work with the network off? One round of chat is not enough - test saying it, remembering it, and being reminded later.
- Where are the model weights? A file already on the device, or a result fetched from a remote machine every time?
- Is an account or an API key mandatory? Some "local apps" are only a local interface; the thinking still happens in the cloud and stops the moment the key is gone.
- Is there a second copy? Logs, diagnostics, automatic sync and crash reports all count.
- Can I see it, edit it and delete it myself? If you cannot see it, "you decide" is not a real claim.
None of these are specific to voice. Swap in files, images or chat history and they still hold.
Two recent numbers say this is no longer theoretical
On November 10, Google added on-device ASR and TTS to AI Edge Gallery. The official post uses the phrase "completely offline": pair lightweight speech models with small language models already running on the device - Gemini Nano on Android, Gemma 4 on other platforms - and you get live transcription alongside locally synthesized replies. The Android path uses ML Kit, supports more than 100 languages and transcribes at 10x real time; its few-step diffusion TTS starts speaking in roughly 100 ms, outputs 24 kHz audio and synthesizes at up to 50x real time, with the models integrated into the OS so they add 0 MB to your APK. The cross-platform path ships open-weight bundles such as Parakeet-TDT 0.6B and PocketTTS 100M.
On November 9, Zyphra released ZONOS3-Mini: a sparse mixture-of-experts model with 100 million active and 800 million total parameters, Apache 2.0, running in a browser through WebGPU. Its published time-to-first-audio is under 570 ms on a Pixel 9 and under 150 ms on an M3 MacBook Air.
Together those two say the question has changed. Keeping listening and speaking on the device is no longer a matter of whether it is possible, but of latency and memory. What remains is not a technical problem: it is whether you have any way to confirm that nothing left.
Walk one sentence through

Before bed you say: "Note the three things I need to follow up on tomorrow and remind me at seven."
If the chain is genuinely local, it runs like this: the microphone takes in audio, on-device transcription turns it into text, a small local model picks out "three things" and "seven tomorrow", the note is written to storage on this device, and a timer pushes it back at the right hour. At each step you can ask the same question - did this step leave the machine? Five answers of "no" are what complete means.
With another kind of product, the chain is: audio uploaded, transcription in the cloud, understanding in the cloud, result stored in an account, notification delivered from a server. It may work very well, and it may even be smarter. It is simply a different answer, and should not be described with the word "local".
The step people skip is the writing-down part. Many demos prove the first half works offline while the record itself syncs to an account. That second half is the one worth testing: is the reminder from three months ago still on this machine today, and can you delete it yourself?
That is the difference between an always-on machine and a tool that only exists while you have it open. KAIHE AIBOX(铠盒AIBOX) keeps private data and conversation history on the device instead of in the cloud, so the easily-overlooked "write it down" step lands locally without extra setup. It does not make the model smarter. It gives that sentence a place to stay that will not wander off.
What it does not do
- Local does not mean never online. Fetching models, updating the system or forwarding a message may still require a request, and those requests should be stated plainly.
- Local does not mean no key. Using a cloud model still requires an API key and still costs money; what survives a network cut is the local part of the workload.
- A small on-device model is not a large cloud model. Live external information and long, heavy reasoning are not what a low-power box is for.
- Local storage is not a backup. Back up the disk anyway.
- There are no real user measurements in this article. The walkthrough above is a scenario, not a benchmark; the latency and parameter figures come from the two official release posts.
Where it fits, and where it does not
It fits low-latency voice interaction, family and work material you would rather not upload, unreliable or absent networks, and the small things you want remembered for months without leaving them in somebody else's account.
It does not fit searches that need live outside information, large-scale reasoning, or transactions that only a platform can complete - booking a flight, for example. Running locally changes nothing about a platform's rules.
Separate the four words, ask the five questions, then decide whether to hand the job over. That is more reliable than checking whether the word "local" appears in the brochure.