Six Ways to Run a Private AI Assistant Without a Mac Mini

Published on: 2026-11-13

Ask anyone how to set up a private AI assistant at home and you get the same answer: buy a Mac mini.

It is a good machine. The current M6 model starts at ¥6,999 with 16GB of unified memory and 153–170GB/s of bandwidth, and it is silent. But it became the default answer mostly because for two years there was no second option worth naming, not because it fits everyone.

So this is not a recommendation. It is a map.

Three numbers decide the outcome

📖 Glossary

AI Box (also known as Agent Computer / Agent PC), is a dedicated local hardware device that runs AI Agents. Pre-installed with an AI agent management system, plug-and-play, running 24/7. Users can remotely command AI to work via Discord, Slack, Telegram, WhatsApp, and more.

Whatever route you take, how well it works comes down to three numbers. Brand, benchmark scores and launch events do not enter into it.

Memory capacity decides what fits. At 4-bit quantization each parameter costs roughly half a byte, so a 7B model is about 4GB, a 14B about 8GB, a 70B around 40GB. If it does not fit, it does not fit. There is no clever workaround.

Memory bandwidth decides how fast it talks. This is the number people skip, and it is where the six routes differ most.

Whether it is still there tomorrow. Something that stays on is an assistant. Something that runs for two hours is a tool.

The arithmetic you can do yourself

KAIHE AIBOX 概念产品与 AGENT ROUTES 全息任务面板

Decode speed works out to roughly bandwidth divided by model size:

tokens per second ≈ bandwidth (GB/s) ÷ model size (GB)

Take a 7B model at 4-bit as the baseline: about 4GB.

A Raspberry Pi 5 has 16GB/s. Sixteen divided by four is four, and measured output is three to five tokens per second. A Mac mini M6 has 170GB/s, which puts it in the low forties, and that matches what people report. An RTX 3090 has 936GB/s, which lands in the hundreds.

This explains something most people have felt: the same model seems dumber on a different machine. The model did not change. The bandwidth did.

It is an order-of-magnitude estimate, not a promise. Quantization format, context length and the inference framework all pull it down, and by how much you have to measure.

Six routes, cheapest first

KAIHE AIBOX 概念产品与 CLOUD MODELS 全息信息层

Your old computer: free. The one to try first, because it answers a more important question than which machine to buy — whether you will actually use one. Install the framework, turn off lid-sleep, run it for two weeks. The cost is battery wear, noise, and the fact that a power cut ends everything.

A Raspberry Pi or similar board: $120 for the 16GB board. 16GB/s of bandwidth. It handles a 7B model at three to five tokens per second, which is fine for a nightly digest or a batch of file sorting, because nobody is waiting. It is not fine for conversation. Asking a question and waiting thirty seconds does not survive three days.

A Mac mini: from ¥6,999. 16GB and 170GB/s. Be precise here: the OS takes four to five of those gigabytes, so a model gets ten or eleven. 7B–8B is comfortable at twenty-odd tokens per second; 14B runs, but weights plus a 16k KV cache land near 14GB, leaving a gigabyte or two, and background apps can push it into swap. 32B needs 18GB and swaps hard. Two things to note: the price excludes a monitor and peripherals, and going to 32GB costs more — that tier starts at ¥12,999.

A unified-memory mini PC: from ¥13,499. The AMD Strix Halo machines. The 128GB version can hand 96GB to the GPU, a capacity no consumer graphics card offers. The catch is bandwidth: 200–270GB/s, more than three times short of a 3090. So it holds a 70B model but does not run it quickly. People who own one are blunt about it: up to 27B with long context is usable, 70B in an agent loop means waiting. The memory is also soldered, so what you buy is what you keep.

A used discrete-GPU desktop: around ¥7,200 for the card. A used 3090 gives 24GB and 936GB/s, five times faster than a Mac mini on the same 7B model. You also need a motherboard, CPU, RAM, PSU and case, the load noise does not belong in a living room, and there is no warranty.

Renting cloud GPU time: tens of yuan per hour. An H100 runs ¥10–21 per hour, occasionally under ¥8 on spot. Under forty hours a week, renting beats buying. But the data leaves, and when you shut it down it is gone — which rules it out for anything meant to be always on.

When the Mac mini is still the right answer

If all three of these hold — you already live in Apple's ecosystem, you want a quiet desktop that doubles as a computer, and you stay at or below 14B — then the ¥6,999 machine is the answer and you can stop reading.

Likewise, if you ask an occasional question or draft the odd document, a browser tab is enough and a dedicated machine is waste. If you only need it during an eight-hour workday, a laptop is fine.

A survey that concludes everything loses to the thing being sold is not a survey.

The route that costs a weekend

All six routes share one hidden cost: turning a machine into something that stays on. Installing the OS, wiring up the runtime, disabling sleep, noticing at midnight that it dropped offline — none of that appears in a price list, but it takes a weekend, and again after every major update.

KAIHE AIBOX(铠盒AIBOX)takes a different route: not bare hardware but a finished unit. System, framework and remote access come preinstalled, it draws 5–12W, and it stays on. One comparison is enough: the A1.

At ¥1,199 with 4GB of memory, it runs the agent layer locally and no local model at all — roughly one sixth of a Mac mini's price. The agent behaves the same, because agent work is not computed locally: it reads files, calls tools, queues tasks, keeps memory, and the reasoning is sent to a cloud model. What the local box has to do is stay awake. 5–12W, passively cooled, no fan, preinstalled, plugged in.

The difference is elsewhere. A Mac mini is also a computer. The A1 is not.

And yes, the Mac mini can run a model locally while the A1 cannot. That deserves precision: 16GB runs a 7B model at four bits, about 4.5GB. The A1 calls models in the hundreds of billions of parameters. Those are not close — a 7B handles classification, summarisation and reformatting, but hand it a long document or someone else's code and it starts inventing. A frontier-scale model does that work daily.

So the ¥6,999 does not buy a stronger AI. It buys a weaker one that runs with the network unplugged — essential in a sealed environment, irrelevant to most people. Reaching cloud-class quality locally means 70B and up with 128GB of memory, which is a bracket twice the Mac mini's price, not an upgrade. So the choice is narrow: if you want a desktop that doubles as a computer, buy the Mac mini; if you want something that is simply always there, the A1 is enough at one sixth the price.

For local inference there are larger units above it. That is a different conversation.

To be clear about the limits: it is neither the cheapest nor the fastest. What it saves is the weekend, and every weekend after that.

Six routes, none of them right for everybody. The question worth answering first is not the budget. It is whether you want to run a model once, or have something that is always there.

Recommended Products

A1 Home Entry A1 Pro Enhanced A2 Professional A2 Pro Advanced X1 Enterprise G1 Flagship
© KAIHE AI - Agent Computer Specialist