한국어EN

Part ReviewPart 2 · How We Got HereDAY 11–20

Part 2 Review

The same ten days, plus the why · one layer deeper · DAY 11–20

For ten days, we traced where AI came from. Here we gather those ten days in one place and, for each day, unpack the 'why' one layer deeper.

No quizzes, no experiments. Each day's first paragraph is the same as in Beginner; the second one goes a step further.

DAY 11 · 1958: the neuron that learns

A neuron machine that learned on its own

In 1958, Frank Rosenblatt built the perceptron. It's one artificial neuron: multiply each signal by a weight, add them up, output 1 if the sum clears a threshold, else 0. The real breakthrough: when wrong, it fixed its own weights — the first machine to learn from examples, not written rules.

Take today's giant AI apart and you'll find huge numbers of these neurons, stacked and wired together. 'Multiply signals by weights and add them up' and 'fix the weights when wrong' — those two ideas haven't changed. McCulloch and Pitts wrote up neurons as logic in 1943, 15 years earlier, but the perceptron was the first real machine to learn on its own.

DAY 12 · Two AI winters

When hopes outrun skill, winter comes

In 1969, Minsky and Papert proved mathematically that one layer of neurons can't solve some problems. Money and people left — the first winter. In the 1980s, expert systems with hand-written rules boomed but failed just off-script; in 1987 their special-computer market crashed — the second winter. Both times, it wasn't that AI was bad — the promises ran ahead of its skill.

The first winter came from the limits of a single layer. Stacking layers would work, but there was no widely used way to tell which weights to fix, and by how much, across layers. A 1986 backpropagation paper in Nature spread the word that stacked layers' weights can be fixed, and the winter began to thaw.

DAY 13 · 2012: deep learning comes back

The day three things came together

In 2012, a University of Toronto team's neural network won the ImageNet photo-guessing contest with a 15.3% error rate. Second place had 26.2%, so the gap was more than 10 percentage points. Hardly anything was new: millions of photos, gaming graphics cards and deep neural networks came together for the first time that year.

ImageNet is a database of millions of labeled photos, released in 2009. Gaming graphics cards do lots of the same math at once, a perfect fit for training neural networks; the winners trained on two for 5–6 days. With every ingredient in place, an old idea finally delivered, and from then on, image research went all-in on neural networks.

DAY 14 · 2017: the Transformer

Words started looking at each other

In 2017, a Google team's paper, 'Attention Is All You Need', set out the Transformer — the backbone of today's AI. Before, sentences were read front to back, one word at a time; the Transformer let every word look at all the others at once. Looking all at once means computing all at once, so it was a perfect match for graphics cards.

In 'The animal didn't cross the street because it was too tired', 'it' is the animal; change it to 'too wide' and 'it' is the street. The Transformer scores how much each word should look at every other word, so it finds links like that in one go. Reading in order, far-apart words got blurry; once words look at each other directly, distance stops mattering.

DAY 15 · 2020: GPT-3

Scaled up, it copied from examples alone

Same Transformer backbone, just made bigger: in 2020 that gave us GPT-3, with 175 billion numbers inside. The surprise: no retraining needed — put two or three examples in the question and it followed along right away. From then on, 'what you tell it to do' — the prompt — started to really matter.

GPT-1 had about 117 million, GPT-2 about 1.5 billion, and GPT-3 175 billion. That's more than 1,000 times bigger between 2018 and 2020. Earlier research had predicted how much performance would rise as the numbers grew; GPT-3 actually pushed that math all the way.

DAY 16 · 2022: ChatGPT

A chat window is what changed the world

ChatGPT, opened to everyone on November 30, 2022, was a GPT-family model put in a back-and-forth chat window. A big part was polishing it with people's picks of the answers they liked better, so it actually got what you asked. So what changed that day was less the tech than who got to use it.

In the 2022 paper on that polishing method, people preferred the answers of a model with 1.3 billion numbers, tuned with human feedback, to those of GPT-3 with 175 billion. It was a sign that size isn't the only answer. Add a window where you just talk — no need to know about adding examples — and only then did everyone get their hands on AI.

DAY 17 · Thinking before answering

Work it out before you answer

AI got much better at hard problems not by getting bigger, but by working through steps before answering. At first, people had to say 'think step by step'; later came reasoning models trained to do it on their own. The catch: longer thinking means slower answers and more text, so it's not worth it for easy questions.

A 2022 paper showed that just adding the line 'Let's think step by step' to the end of a question makes math scores jump. In 2024 came a model trained to write out its working unprompted: on the same US high school math contest, an instant-answer model got 12% and this one got 74%. That came from changing the order of answering, not the backbone.

DAY 18 · AI that uses tools

Grabbing a calculator when you don't know

AI's main job is adding the next word, so it's weak at exact things like big multiplications or today's weather. So it got tools — a calculator, search, running code — and from 2023, models chose by themselves when to grab one. A good AI isn't one that knows everything — it's one that notices it doesn't know and grabs a tool.

It gets the question, decides if it needs a tool, calls the tool in a set format, and finishes the answer with what comes back. Step two is the key — it used to take a person saying 'search for it', but now the model decides by itself. In 2023, the Toolformer paper had a model teach itself when to call a calculator, search or a calendar; that year, models officially got a set format for naming a tool and its values.

DAY 19 · AI that sees and hears

Chop anything into pieces, line them up

AI didn't learn to see photos and hear sounds by getting eyes and ears. Photos are cut into square grid pieces and sound into tiny time slices, then lined up in one row with the text pieces. Then no new backbone is needed — just feed them into the Transformer, where pieces look at each other.

At first, a photo was written out as text separately, then handed to a language model. Models that learned text, images and sound together in one backbone from the start skip that middle step: faster and more accurate. Photo reading opened up widely in 2023; in 2024 a model trained on all three together cut spoken replies to 0.32 seconds on average.

DAY 20 · Everyone's model race

From using one to picking one

In late 2022 you could count good chat AIs on one hand; now dozens of groups in the US, Europe, China and Korea make their own. The biggest change: open models, whose whole bundle of numbers is released for anyone to download and run on their own gear. So now you don't just use one AI — you pick, weighing price, speed and what each one is good at.

Models anyone could download grew from 65 billion in February 2023 to 70 billion that July, then 405 billion in July 2024. That's six times bigger in a year and a half — anyone could now get a model bigger than GPT-3's 175 billion. In 2025, a Chinese research team openly released a reasoning model too — so the open-model race isn't just one country's thing.

Ten days, tied into one line

The story that started with one neuron in 1958 froze twice as hopes ran ahead of skill, then woke up again in 2012 when data, computing power and depth came together.

Once the Transformer set the backbone in 2017, the backbone barely changed. It was scaled up (2020), polished to people's taste and put in a chat window (2022), made to think before answering, handed tools, and given photos and sound in the same row. Most of what changed was 'how it's used'.

As a result, there are now dozens of models with similar skills. In Part 3, we try out what you can actually do with them, one thing at a time — starting with summarizing and translation.

Learn more

  1. Frank Rosenblatt, The Perceptron (1958) — the first artificial neuron that learnsen.wikipedia.org/wiki/Perceptron
  2. McCulloch & Pitts, A logical calculus of the ideas immanent in nervous activity (1943)link.springer.com/article/10.1007/BF02478259
  3. Minsky & Papert, Perceptrons (1969) — the limits of a single layermitpress.mit.edu/9780262630221/perceptrons
  4. Learning representations by back-propagating errors (1986) — backpropagationnature.com/articles/323533a0
  5. ImageNet: A Large-Scale Hierarchical Image Database (2009)image-net.org/static_files/papers/imagenet_cvpr09.p…
  6. ILSVRC 2012 Results — top-5 error 15.3%image-net.org/challenges/LSVRC/2012/results.html
  7. ImageNet Classification with Deep Convolutional Neural Networks (NIPS 2012)papers.nips.cc/paper_files/paper/2012/hash/c399862d…
  8. Attention Is All You Need (2017) — the Transformerarxiv.org/abs/1706.03762
  9. Language Models are Few-Shot Learners (2020) — GPT-3, 175 billion parametersarxiv.org/abs/2005.14165
  10. Introducing ChatGPT (OpenAI, 2022)openai.com/index/chatgpt
  11. Training language models to follow instructions with human feedback (2022) — RLHFarxiv.org/abs/2203.02155
  12. Large Language Models are Zero-Shot Reasoners (2022) — 'Let's think step by step'arxiv.org/abs/2205.11916
  13. Learning to Reason with LLMs (OpenAI, 2024) — 12% and 74%openai.com/index/learning-to-reason-with-llms
  14. Toolformer (2023) — learning on its own when to call a toolarxiv.org/abs/2302.04761
  15. Function calling and other API updates (OpenAI, 2023)openai.com/index/function-calling-and-other-api-upd…
  16. GPT-4V(ision) System Card (OpenAI, 2023)openai.com/index/gpt-4v-system-card
  17. Hello GPT-4o (OpenAI, 2024) — voice replies in 0.32 seconds on averageopenai.com/index/hello-gpt-4o
  18. Introducing LLaMA (Meta AI, 2023) — 65 billion, openly releasedai.meta.com/blog/large-language-model-llama-meta-ai
  19. The Llama 3 Herd of Models (2024) — 405 billion, openly releasedarxiv.org/abs/2407.21783
  20. DeepSeek-R1 (2025) — an open reasoning modelarxiv.org/abs/2501.12948