vLLM details Hybrid HiSparse offloading in GLM 5.3 optimization part 1
Released Hybrid HiSparse offloading as part 1 of its GLM 5.3 optimization series. HiSparse is a pressure-driven memory hierarchy that keeps frequently accessed tokens on the GPU and moves cold data down to the CPU; offloading kicks in only when GPU memory actually runs short, so decoding continues without preempting requests. It targets serving a context of 1 million tokens on a node of 8 H200s, and the company claims that on a 13-turn OpenHands agent workload averaging 74,160 tokens in the initial turn and 753 tokens in follow-up turns, a configuration of a 384GiB HiSparse pool + a 128GiB offloading pool delivered higher concurrent throughput than the existing 512GiB offloading. States that it had not been released at the time of posting and is slated for general availability in vLLM v0.30.
Why it matters
An attempt to handle contexts in the 1 million-token class on a single node; solutions to the memory bottleneck of long-context agents are converging on a combination of sparse attention and hierarchical memory.
Source — Primary sources first; secondary reports are labelled “Press”
Officialvllm.aihttps://vllm.ai/blog/2026-09-08-glm53-part1-hybrid-sparse-offloadingAdded in Week 38, 2026 · See that week