⑲6 records
On-Device & Local Inference
- Aug 31, 2026HUMAIN, Qualcomm — Unveiled the Horizon Ultra AI PC at LEAP 2026 (Riyadh) — based on the Snapdragon X2 Elite (18-core Oryon CPU, Adreno GPU, Hexagon NPU), it spreads AI work across the CPU, GPU and NPU. Enterprise purchases of the Windows version begin on Sep 20, 2026; a HUMAIN OS version is planned for 2027. The press release gives no NPU TOPS figure or price SourceA Saudi sovereign wealth fund affiliate puts forward an on-device AI PC as the endpoint of a sovereign AI stack — a phase in which local inference becomes an item of geopolitical procurement
- May 19, 2026Google (AI Edge) — LiteRT-LM generally available (GA) — added a native Swift API for iOS and a WebGPU-accelerated JavaScript API. Supports Android CPU, OpenCL GPU and NPU, iOS Metal and browser WebGPU. With Gemma 4: 52 tok/s decode on Android GPU, 56 tok/s on iOS GPU, 76 tok/s on the web on a MacBook Pro M4 Max, up to 2.2× with Multi-Token Prediction, 607MB of memory on iOS SourceThe stage at which an on-device runtime covers not just mobile OSes but browsers with the same stack. Figures are the vendor's own measurements
- May 5, 2026Google (Chrome) — Opened the Prompt API to the web in Chrome 148 stable — for extensions it has been stable since Chrome 138. Web pages access the browser's built-in Gemini Nano directly, with support for text, image and audio input and for constraining responses with regular expressions and JSON schemas. Requirements: 22GB of free space, and more than 4GB of VRAM or 16GB of RAM and 4 or more cores SourceA stable-channel case of an on-device model exposed as a web platform API rather than an app. Responsibility for model distribution and storage management passes to the browser vendor
- Nov 28, 2025Liquid AI — Published the LFM2 technical report — expanded to a 350M–8.3B family (dense 350M, 700M, 1.2B, 2.6B; MoE 8.3B total / 1.5B active), pretrained on 10–12T tokens, 32K context. Describes the design as coming from a hardware-in-the-loop architecture search under edge latency and memory constraints. LFM2-2.6B scored 79.56% on IFEval and 82.41% on GSM8K SourceA follow-up document to the July release. A case of an on-device family having public documentation at the level of a technical report
- Sep 10, 2025Arm — Lumex CSS platform announced — a C1 CPU cluster with SME2, and the Mali G1-Ultra. KleidiAI integrated into PyTorch ExecuTorch, Google LiteRT, Alibaba MNN and Microsoft ONNX Runtime. Claims up to 5× AI performance, 4.7× lower latency on speech workloads and 2.8× faster audio generation SourceAn approach that tackles the on-device inference bottleneck with a CPU instruction-set extension (SME2) rather than a dedicated NPU. Framework integrations were announced at the same time, so it is deployed through the software path, not through silicon. The figures are the vendor's own claims
- Jul 10, 2025Liquid AI — LFM2 released — 350M, 700M and 1.2B dense checkpoints, announced with a hybrid backbone that mixes gated short convolutions with a small number of GQA blocks. License based on Apache 2.0 (commercial use allowed under $10 million in annual revenue). Claims 2× the CPU decode and prefill speed of Qwen3, measured with ExecuTorch and llama.cpp on a Galaxy S24 Ultra and an AMD Ryzen HX370 SourceA commercial model family that treats on-device as a design goal rather than a by-product of post-hoc quantization, searching architectures with hardware in the loop. The figures are the vendor's own measurements, not independently checked