What is a diffusion language model (dLLM)? A language model that fills in blanks over several rounds instead of one token at a time

A language model that fills in blanks over several rounds instead of one token at a time.

A diffusion language model doesn't write one token at a time from left to right. It lays out the whole answer at once and improves it over several rounds, like an image diffusion model clearing noise step by step.

Models like LLaDA start with every position in the answer hidden; at each step they guess hidden tokens and fill several blanks together. Training works the same way, hiding parts of a text to guess. Making several tokens at once can speed things up.

In June 2025, Inception Labs said Mercury Coder Mini produced 1,109 tokens per second on NVIDIA H100 GPUs. A weak spot under study: blanks filled separately at once can end up with words that don't fit together.

Think of it like this

In a dictation test you write from the first letter to the last, but in a crossword you fill the easy squares first and use those letters as clues for the rest. A diffusion language model finishes text the crossword way.

For example

It might start as '□ □ □ □ □', become '□ □ for □ □' after one step, and then 'We went for some lunch' in the next, with several blanks filled at once.

Common mix-ups

Mix-upAn AI can only ever write from left to right.
ActuallyLLaDA, published in February 2025, is an 8-billion-parameter diffusion language model that held its own against LLaMA3 8B at learning from examples in the prompt. Given a line from a famous Chinese poem and asked for the line before it, it beat GPT-4o, though GPT-4o was far better at giving the next line.
Mix-upA diffusion language model has to be trained completely from scratch.
ActuallyIn July 2026, Google DeepMind made DiffusionGemma by fine-tuning the existing Gemma 4 instead of training from scratch. It took less than 10% of the tokens used to train the original model.

Try it in class

For teachers at the start of a lesson — or for students working on their own.

Question: Writing in order from the start, or filling all the blanks at once: which suits which kind of text?
Activity: Pick a topic such as today's school lunch and make a card with six blanks. Two group members fill one blank each at the same time without talking, finishing in three rounds; then one person writes a sentence one word at a time from the start, in six rounds. Compare the number of steps and the awkward spots, then read the Diffusion-LM (E0154), Mercury (E0617) and DiffusionGemma (E1025) records in order to trace how this line of work grew.