한국어EN

Part ReviewPart 2 · How We Got HereDAY 11–20

Part 2 Review

The real terms, defined and then used · down to the mechanics and backstory · DAY 11–20

For ten days, we traced where AI came from. Here we gather those ten days in one place and, for each day, go into how it works and the real terms.

No quizzes, no experiments. Each new term gets a one-sentence explanation the first time it shows up; after that, we just use its name.

DAY 11 · 1958: the neuron that learns

A neuron machine that learned on its own

In 1958, Frank Rosenblatt built the perceptron. It's one artificial neuron: multiply each signal by a weight, add them up, output 1 if the sum clears a threshold, else 0. The real breakthrough: when wrong, it fixed its own weights — the first machine to learn from examples, not written rules.

Take today's giant AI apart and you'll find huge numbers of these neurons, stacked and wired together. 'Multiply signals by weights and add them up' and 'fix the weights when wrong' — those two ideas haven't changed. McCulloch and Pitts wrote up neurons as logic in 1943, 15 years earlier, but the perceptron was the first real machine to learn on its own.

In the real terms, the perceptron multiplies each input by its weight and adds them up (the weighted sum), adds a bias, and a step function outputs 1 if that's above 0. The learning rule is simple — when it gets an example wrong, nudge the weights a little toward the right answer. A one-layer model like this only learns properly if one straight line can split the data — a condition called linearly separable, and exactly where the next day's story starts.

DAY 12 · Two AI winters

When hopes outrun skill, winter comes

In 1969, Minsky and Papert proved mathematically that one layer of neurons can't solve some problems. Money and people left — the first winter. In the 1980s, expert systems with hand-written rules boomed but failed just off-script; in 1987 their special-computer market crashed — the second winter. Both times, it wasn't that AI was bad — the promises ran ahead of its skill.

The first winter came from the limits of a single layer. Stacking layers would work, but there was no widely used way to tell which weights to fix, and by how much, across layers. A 1986 backpropagation paper in Nature spread the word that stacked layers' weights can be fixed, and the winter began to thaw.

The classic problem one layer can't solve is XOR — output 1 when exactly one of two inputs is 1; no straight line splits its four points, so it isn't linearly separable. Adding just one more layer solves it, but then you have to work out how much each middle-layer weight is to blame for the error. Backpropagation uses the chain rule from calculus to hand the output's error backward, starting from the last layer, and nearly all neural network training today stands on it.

DAY 13 · 2012: deep learning comes back

The day three things came together

In 2012, a University of Toronto team's neural network won the ImageNet photo-guessing contest with a 15.3% error rate. Second place had 26.2%, so the gap was more than 10 percentage points. Hardly anything was new: millions of photos, gaming graphics cards and deep neural networks came together for the first time that year.

ImageNet is a database of millions of labeled photos, released in 2009. Gaming graphics cards do lots of the same math at once, a perfect fit for training neural networks; the winners trained on two for 5–6 days. With every ingredient in place, an old idea finally delivered, and from then on, image research went all-in on neural networks.

The winner was a convolutional neural network (CNN), strong on grid-shaped input: it slides small filters across the whole photo to find features like edges and patterns. The deeper the stack, the more the features build up: early layers spot lines and edges, later ones spot bigger chunks like eyes or wheels. 'Top-5 error' is how often the right answer isn't among the model's five guesses, so 15.3% means the answer was in its top five for about 85 of every 100 photos.

DAY 14 · 2017: the Transformer

Words started looking at each other

In 2017, a Google team's paper, 'Attention Is All You Need', set out the Transformer — the backbone of today's AI. Before, sentences were read front to back, one word at a time; the Transformer let every word look at all the others at once. Looking all at once means computing all at once, so it was a perfect match for graphics cards.

In 'The animal didn't cross the street because it was too tired', 'it' is the animal; change it to 'too wide' and 'it' is the street. The Transformer scores how much each word should look at every other word, so it finds links like that in one go. Reading in order, far-apart words got blurry; once words look at each other directly, distance stops mattering.

This setup is called self-attention. Each word makes three vectors (lists of numbers): a query ('what I want'), a key ('what I have') and a value ('what I hand over'); the better a query fits a key, the more of that word's value it takes. The recurrent neural networks (RNNs) before it had to finish one word before the next, so they were hard to parallelize; self-attention computes every pair at once with matrix multiplication — what graphics cards do best.

DAY 15 · 2020: GPT-3

Scaled up, it copied from examples alone

Same Transformer backbone, just made bigger: in 2020 that gave us GPT-3, with 175 billion numbers inside. The surprise: no retraining needed — put two or three examples in the question and it followed along right away. From then on, 'what you tell it to do' — the prompt — started to really matter.

GPT-1 had about 117 million, GPT-2 about 1.5 billion, and GPT-3 175 billion. That's more than 1,000 times bigger between 2018 and 2020. Earlier research had predicted how much performance would rise as the numbers grew; GPT-3 actually pushed that math all the way.

GPT-3's trick is called in-context learning — without changing a single weight, it picks up the rule on the spot just from the examples in its input. Depending on how many examples you give, it's called zero-shot (0), one-shot (1) or few-shot (a few) — that's the 'Few-Shot' in the paper's title. The finding that performance improves along a predictable curve as model size, data and compute grow is called the scaling laws — it became the case for investing in 'make it bigger'.

DAY 16 · 2022: ChatGPT

A chat window is what changed the world

ChatGPT, opened to everyone on November 30, 2022, was a GPT-family model put in a back-and-forth chat window. A big part was polishing it with people's picks of the answers they liked better, so it actually got what you asked. So what changed that day was less the tech than who got to use it.

In the 2022 paper on that polishing method, people preferred the answers of a model with 1.3 billion numbers, tuned with human feedback, to those of GPT-3 with 175 billion. It was a sign that size isn't the only answer. Add a window where you just talk — no need to know about adding examples — and only then did everyone get their hands on AI.

Polishing with human feedback is called RLHF (reinforcement learning from human feedback). First it's tuned on model answers people wrote; then people rank several answers to one question, that data builds a reward model scoring 'which answer is better', and the language model is retrained to push that score up. Getting a model to act the way people intend like this is called alignment, and research into making answers useful and harmless really got going here.

DAY 17 · Thinking before answering

Work it out before you answer

AI got much better at hard problems not by getting bigger, but by working through steps before answering. At first, people had to say 'think step by step'; later came reasoning models trained to do it on their own. The catch: longer thinking means slower answers and more text, so it's not worth it for easy questions.

A 2022 paper showed that just adding the line 'Let's think step by step' to the end of a question makes math scores jump. In 2024 came a model trained to write out its working unprompted: on the same US high school math contest, an instant-answer model got 12% and this one got 74%. That came from changing the order of answering, not the backbone.

Having it write out the working first is called chain-of-thought. A language model writes one token at a time, so writing down in-between results lets it look back at them when picking the next token — scratch paper instead of mental math. Reasoning models tend to solve more hard problems the more they think before answering; this compute spent at answer time is called test-time compute, and it became a second dial besides making the model bigger.

DAY 18 · AI that uses tools

Grabbing a calculator when you don't know

AI's main job is adding the next word, so it's weak at exact things like big multiplications or today's weather. So it got tools — a calculator, search, running code — and from 2023, models chose by themselves when to grab one. A good AI isn't one that knows everything — it's one that notices it doesn't know and grabs a tool.

It gets the question, decides if it needs a tool, calls the tool in a set format, and finishes the answer with what comes back. Step two is the key — it used to take a person saying 'search for it', but now the model decides by itself. In 2023, the Toolformer paper had a model teach itself when to call a calculator, search or a calendar; that year, models officially got a set format for naming a tool and its values.

This format for a model to call a tool is called function calling. The model just writes 'which function, with which values' as text in a preset structure; an outside program runs it and feeds the result back — the model isn't pressing calculator buttons itself. A system that chains these steps many times — planning on its own, calling tools and checking the results — is called an agent, and most of today's 'hand it a task' AI is built this way.

DAY 19 · AI that sees and hears

Chop anything into pieces, line them up

AI didn't learn to see photos and hear sounds by getting eyes and ears. Photos are cut into square grid pieces and sound into tiny time slices, then lined up in one row with the text pieces. Then no new backbone is needed — just feed them into the Transformer, where pieces look at each other.

At first, a photo was written out as text separately, then handed to a language model. Models that learned text, images and sound together in one backbone from the start skip that middle step: faster and more accurate. Photo reading opened up widely in 2023; in 2024 a model trained on all three together cut spoken replies to 0.32 seconds on average.

The square pieces you get by cutting a photo into a grid are called patches. Turn each piece into a list of numbers and tag it with where it sat in the photo, and to the Transformer it's no different from a word token. Handling different kinds of input together like this — text, images, sound — is called multimodal; speed and accuracy depend on whether a model learned them together from the start or had separate parts joined up.

DAY 20 · Everyone's model race

From using one to picking one

In late 2022 you could count good chat AIs on one hand; now dozens of groups in the US, Europe, China and Korea make their own. The biggest change: open models, whose whole bundle of numbers is released for anyone to download and run on their own gear. So now you don't just use one AI — you pick, weighing price, speed and what each one is good at.

Models anyone could download grew from 65 billion in February 2023 to 70 billion that July, then 405 billion in July 2024. That's six times bigger in a year and a half — anyone could now get a model bigger than GPT-3's 175 billion. In 2025, a Chinese research team openly released a reasoning model too — so the open-model race isn't just one country's thing.

The 'bundle of numbers' here is the full set of weights fixed by training, and a model that releases them is called an open-weight model. With open weights, anyone can do extra training on their own data (fine-tuning), or shrink the model down to run on a laptop. But opening the weights doesn't mean opening the training data or code too, so to see how far 'open' goes, you have to read each model's license.

Ten days, tied into one line

The perceptron's 'weighted sum and weight updates' (1958) are still with us; its limit (XOR) was solved by stacking layers and backpropagation; and deep neural networks finally delivered in 2012, once data and GPUs were in place.

After self-attention set a backbone that computes in parallel in 2017, scaling (2020), RLHF (2022), chain-of-thought and test-time compute, function calling, and multimodal were added on, one after another. Most of them changed how models are trained and used, not the backbone.

And as open weights spread, all of this went from a few labs' tech to everyone's options. In Part 3, we try out what you can actually do with them, one thing at a time — starting with summarizing and translation.

Learn more

  1. Frank Rosenblatt, The Perceptron (1958) — the first artificial neuron that learnsen.wikipedia.org/wiki/Perceptron
  2. McCulloch & Pitts, A logical calculus of the ideas immanent in nervous activity (1943)link.springer.com/article/10.1007/BF02478259
  3. Minsky & Papert, Perceptrons (1969) — the limits of a single layermitpress.mit.edu/9780262630221/perceptrons
  4. Learning representations by back-propagating errors (1986) — backpropagationnature.com/articles/323533a0
  5. ImageNet: A Large-Scale Hierarchical Image Database (2009)image-net.org/static_files/papers/imagenet_cvpr09.p…
  6. ILSVRC 2012 Results — top-5 error 15.3%image-net.org/challenges/LSVRC/2012/results.html
  7. ImageNet Classification with Deep Convolutional Neural Networks (NIPS 2012)papers.nips.cc/paper_files/paper/2012/hash/c399862d…
  8. Attention Is All You Need (2017) — the Transformerarxiv.org/abs/1706.03762
  9. Language Models are Few-Shot Learners (2020) — GPT-3, 175 billion parametersarxiv.org/abs/2005.14165
  10. Introducing ChatGPT (OpenAI, 2022)openai.com/index/chatgpt
  11. Training language models to follow instructions with human feedback (2022) — RLHFarxiv.org/abs/2203.02155
  12. Large Language Models are Zero-Shot Reasoners (2022) — 'Let's think step by step'arxiv.org/abs/2205.11916
  13. Learning to Reason with LLMs (OpenAI, 2024) — 12% and 74%openai.com/index/learning-to-reason-with-llms
  14. Toolformer (2023) — learning on its own when to call a toolarxiv.org/abs/2302.04761
  15. Function calling and other API updates (OpenAI, 2023)openai.com/index/function-calling-and-other-api-upd…
  16. GPT-4V(ision) System Card (OpenAI, 2023)openai.com/index/gpt-4v-system-card
  17. Hello GPT-4o (OpenAI, 2024) — voice replies in 0.32 seconds on averageopenai.com/index/hello-gpt-4o
  18. Introducing LLaMA (Meta AI, 2023) — 65 billion, openly releasedai.meta.com/blog/large-language-model-llama-meta-ai
  19. The Llama 3 Herd of Models (2024) — 405 billion, openly releasedarxiv.org/abs/2407.21783
  20. DeepSeek-R1 (2025) — an open reasoning modelarxiv.org/abs/2501.12948