QQuestion: How did chatbots learn to talk like people?

It didn't happen all at once — it took several steps, one after another. Underneath them all is a simple exercise: look at the text so far and guess what comes next.

First, in 2003, Yoshua Bengio and colleagues proposed a language model built from a neural network. It learned words as lists of numbers (vectors), and it became the direct forerunner of later work that turns words into numbers¹.

In June 2017, Google researchers released the Transformer, which made it possible to train on a whole sentence at once instead of reading it one word at a time. That made training on huge amounts of text practical². GPT-3 (175B), from May 2020, was a model scaled up this way, and it could take on a new task just from examples in the prompt, without being trained again³.

But continuing text is not the same as following a person's request. In January 2022, OpenAI released InstructGPT and wrote that it unlocked abilities that had been hard to draw out through careful prompting alone⁴. In the paper released in March 2022, people preferred the answers of a 1.3B model, tuned on records of which answers people picked as better, to those of the 175B GPT-3. That method, RLHF, became the standard way to tune AI⁵. Then on November 30, 2022, ChatGPT came out as a chat window, and anyone — not just experts — could talk with an AI⁶.

So a chatbot's way with words comes from practicing next-word guessing on enormous amounts of text, then being tuned with answers people chose. Keep in mind, too, that talking well is not the same as always being right.