Diffusion Language Models
8 records · 0 notes · records tagged with this as a related axis are included.
Not one token at a time, but polishing the whole passage
Most language models today write one token after another, from the start. Diffusion language models start from noise and make text by revising the whole passage over several rounds. This axis takes in research that carried the method established in image generation over to text, commercial models said to be fast because they make many tokens at once, and research fixing the weak spots that come with that. It is still an axis with few records, so for speed claims the first thing noted is whether the company measured the figure itself.
AIMade by AIThis introduction was written by AI (Claude) from the axis's records, and published after automated checks. The owner doesn't read every item.How it's made
The point at which generation paths outside autoregression matured from the claim that "parallel generation is fast" to the question of "how to preserve dependency structure". A design that tries to solve the respective
A case of solving, through architecture, a chronic limitation of diffusion language models: "state breaks at chunk boundaries". A substantive advance that arrived just as the observation topic "diffusion language models"
A case of the diffusion language model family crossing over into image generation, adding to the accumulated evidence for the observation topic "diffusion language models". In the authors' own reporting, it claims 53.53
A strand that, instead of the ColPali lineage of reusing generative models as retrievers, returns to a single tower designed as an encoder from the start — a claim that "encoder-only" is valid again for multimodal retrie
Shows a path to obtaining a diffusion model with less than 10% of the original token budget; about 1,500 tok/s on a single H100
Claims up to 10× the speed of speed-optimized frontier models. The first time a diffusion LM entered a commercial track
Reopened the question of whether autoregression is a necessary condition for LLM capabilities at all
The starting point of the diffusion language model lineage