Researchers release Random Attention
Random Attention released — against the usual approach to KV-cache eviction at inference time, which scores cached tokens to decide which ones to keep, reports that the selection signal contributes almost nothing, and presents a method that preserves the prompt while removing tokens at random within each attention head, eliminating score computation altogether. Reports that across 4 models and 6 reasoning benchmarks it matches stronger alternatives in performance while delivering considerably higher throughput in a real vLLM setting
Why it matters
Raises a counterargument against the entire lineage of importance-score-based KV eviction: "the scores themselves are unnecessary". The authors explain that only the prompt is fragile, while reasoning sequences sustain themselves through a double redundancy — textual restatement and duplication across multiple attention heads — so as long as the prompt is protected, random selection is enough. Figures are as reported by the authors.
Source — Primary sources first; secondary reports are labelled “Press”
Paperarxiv.orghttps://arxiv.org/abs/2609.03430Added in Week 36, 2026 · See that week