vLLM publishes optimization results for SemiAnalysis's AgentX benchmark
Announced optimization results tuned for AgentX, a public benchmark built on real agentic coding traces from SemiAnalysis. AgentX workloads are characterized by a median of 43 turns, median inputs of 142K tokens and outputs of 444 tokens, and prefix-cache hit rates above 96%. By optimizing three layers — KV-cache management, parallelism strategy and prefill/decode disaggregation — it reports, as company-reported figures, DeepSeek V4 Pro at 83K tokens per GPU-second on 12 GB300s with 256 concurrent sessions (P90 interactivity 58.3 tok/s), MiniMax M3 at 70K TPGS on 2 B300s (74.2 tok/s) and Kimi K3 at 11.8K TPGS on 16 GB300s (62.7 tok/s). It claims serving costs 106, 85 and 14.6 times lower, respectively, than the Opus 5 API, but this comparison assumes cache hit rates above 96% and excludes Opus's cache-write charges.
Why it matters
As agentic coding becomes the representative workload of inference serving, the goal of benchmarks itself has shifted from one-shot throughput to the economics of long multi-turn sessions.
Source — Primary sources first; secondary reports are labelled “Press”
Officialvllm.aihttps://vllm.ai/blog/2026-09-08-vllm-agentxAdded in Week 38, 2026 · See that week