The neural network design behind most of today's large language models.
The Transformer is a neural network design introduced by a Google team in the 2017 paper "Attention Is All You Need". Most of today's large language models, like GPT, Claude and Gemini, are built on it.
Older designs usually read a sentence one word at a time, in order. A Transformer uses attention so every token can look at every other token at once, which makes it easier to link words that sit far apart.
Looking at everything at once also means computing it all at once, which suits GPUs. That let researchers train on huge amounts of text and scale models up.
Think of it like this
In the telephone game, a message passes from person to person and gets fuzzy by the end of the line. A Transformer is more like a class group chat: everyone sees every message at once, so it's easy to tell who is replying to whom.
For example
In "Sam lent Maya his notebook, and it was brand new," the model works out that "it" means the notebook.
Records about this
Source-checked moments from AI history.
Self-attention alone, without recurrence, made parallel training possible, establishing the architectural premise for large-scale pretraining