A local AI design that handles prefill and decode on different chips
The 2026 Mac mini has been reshaped into an edge node aimed at running local LLMs, and behind the rise in its base price from $599 to $899 are foundry economics in which TSMC's 2 nm wafer cost jumped 66% to $30,000 per wafer. The core frame is splitting the two stages: prefill (up to the first token, compute-bound) is handled by the M6's GAA nanosheets and the neural accelerators built into each GPU core, making prompt processing 4.8 times faster, while decode (token generation, memory-bandwidth-bound) favors the M5 Pro's 307 GB/s of bandwidth. However, the M5 Pro is fixed at 64 GB of unified memory, so loading a 70B-class model whose weights alone take about 45 GB — plus the KV cache and OS memory — starts swapping, and speed collapses to under 1 word per second. The workaround is clustering, linking several machines over Thunderbolt 5 RDMA to pool their memory, and product specs such as prices and code names need to be checked against official announcements.
Key points
- A 66% rise in 2 nm wafer costs ($30,000 per wafer) feeds straight into a $300 price increase
- Prefill is compute-bound and decode is memory-bandwidth-bound — the bottlenecks differ
- At the 64 GB ceiling, a 70B model collapses to under 1 word per second from swapping
- Thunderbolt 5 RDMA clustering cuts latency from 300 µs to under 3 µs and pools memory
This is a note: my own write-up after reading several sources. It has no source link; events checked against a source are kept as records.