Researchers propose register tokens to pass reasoning state in diffusion LMs
Proposed a method that gives masked diffusion language models fixed-size "register tokens" to pass reasoning state between chunks. Compared with passing state as discrete text, it gains up to +8.5 points on math and up to +19.5 points on code, and leads on every benchmark for both LLaDA and Dream. Because generation can continue even after earlier content is discarded, it is especially effective for code generation spanning multiple chunks, and it can be refined further with reinforcement learning on long-horizon reasoning tasks.
Why it matters
A case of solving, through architecture, a chronic limitation of diffusion language models: "state breaks at chunk boundaries". A substantive advance that arrived just as the observation topic "diffusion language models" passed 10 records in total.
Source — Primary sources first; secondary reports are labelled “Press”
Paperarxiv.orghttps://arxiv.org/abs/2609.16372Added in Week 39, 2026 · See that week