Part 2: How We Got Here

Where did AI start?

In 1958, Frank Rosenblatt built a machine called the perceptron — an artificial neuron copying a single nerve cell in the human brain.

It does just one thing. It multiplies each incoming signal by a weight, adds them all up, and if the total clears a threshold it outputs 1 — if not, 0.

Here's what was really new — when it got something wrong, it nudged its own weights a little. It was the first machine that learned just from examples, without people writing the rules.

Try it — move the weights to clear the threshold

This artificial neuron decides whether to watch an online clip. Switch the signals on and off up top, and drag the weights below. If the total clears the threshold of 120 (the orange line), you get 1; if not, 0.

In one picture

Multiply, add, compare with the threshold — that's all one artificial neuron does.

One artificial neuron — multiply, add, compare with the threshold.

1. Q1What does one perceptron (artificial neuron) do, in one line?

   1. ① Memorizes whole sentences and repeats them
   2. ② Multiplies by weights, adds up, 1 past threshold
   3. ③ Searches the internet for the answer
   4. ④ Follows rules people wrote, one by one

2. Q2What made the 1958 perceptron most different from earlier machines?

   1. ① It computed much faster
   2. ② It didn't use electricity
   3. ③ It fixed its own weights when wrong
   4. ④ It could make pictures and music

3. Q3What else do we call the perceptron's ‘weights’ in today's AI?

   1. ① Tokens
   2. ② Parameters — from DAY 05
   3. ③ Prompts
   4. ④ Temperature

Today's AI is really just that one 1958 neuron, stacked up many, many times.

Why did AI freeze over twice?

In 1969, while yesterday's perceptron was riding high, Minsky and Papert proved with math that there are problems a single layer of neurons can't solve. Funding and people drained away — that was the first winter.

In the 1980s, expert systems — where people wrote out every rule by hand — were all the rage. But step even a little outside the situations they'd been given, and they got it wrong. In 1987, the market for their special computers collapsed completely — the second winter.

Both times the reason was the same. AI didn't suddenly get dumb — the promises had gotten ahead of the skills.

Try it — put the winters in order

You know that game with an amazing trailer that turned out kind of meh once you played it? Then everyone bails. That's exactly what an AI winter is. Put the three stories in order.

In one picture

Hopes shot up twice, and crashed both times. The gray bands are the winters.

Two winters — the peaks were always built by promises, and the valleys always came when the promises weren't kept.

1. Q1What did Minsky and Papert's 1969 book prove?

   1. ① Computers calculate faster than people
   2. ② Some problems a one-layer perceptron can't solve
   3. ③ Bigger neural nets are always better
   4. ④ You should never give AI written rules

2. Q2What was the biggest reason 1980s expert systems collapsed?

   1. ① They used too much electricity
   2. ② Rules were hand-written, so new cases broke them
   3. ③ There was no internet yet
   4. ④ Nobody was interested

3. Q3What do both AI winters teach us?

   1. ① AI just doesn't work
   2. ② Overpromising brings winter
   3. ③ Money alone always wins
   4. ④ New theories bring winter

Winter didn't come because AI was bad — it came because the promises got ahead of the skills.

One photo contest flipped the whole game

In the 2012 ImageNet photo-recognition contest, a neural network from a University of Toronto team won with a top-5 error of 15.3%. Second place was 26.2%, so the gap was more than 10 percentage points — a gap that big was a first for that contest.

Almost nothing was newly invented. Millions of photos · gaming graphics cards · a deeply stacked neural network — that year, those three came together for the first time.

After that, research on images switched over to neural networks across the board. It was the day the second winter from yesterday truly ended.

Try it — before and after 2012

It's just like watching a before-and-after video of a game patch. Pick a topic, then switch between “Up to 2011” and “2012.” You'll see right away what changed.

In one picture

The error rate had been shrinking a little each year — then in 2012, one entry suddenly plunged.

In 2012 the winner's error rate beat second place by more than 10 percentage points — a first for that contest.

1. Q1Why was the 2012 ImageNet result such a shock?

   1. ① The winner hit 0% error, a first ever
   2. ② Winner 15.3% vs. 2nd 26.2%: a 10+ point gap
   3. ③ People entered the contest for the first time
   4. ④ Only one team entered that year

2. Q2What's the biggest reason deep learning only took off in 2012?

   1. ① Neural networks were invented that year
   2. ② Photos, graphics cards, deep nets finally met
   3. ③ The internet was invented that year
   4. ④ The government made AI research mandatory

3. Q3Why were gaming graphics cards (GPUs) used for deep learning?

   1. ① They make screens look nice
   2. ② They do tons of math at once
   3. ③ The internet got faster
   4. ④ They have lots of storage

What ended the winter wasn't a new idea — it was the day data, computing and depth came together.

The words in a sentence started looking at each other

In 2017, one paper from a Google research team set the basic design of the AI you use today. The design is called the Transformer, and the paper's title is 'Attention Is All You Need'.

Before that, AI read a sentence one word at a time, from the start. The Transformer let every word in a sentence look at all the others at the same time — that's called 'attention.'

Looking at everything at once means it can all be calculated in one go. That made it a perfect fit for graphics cards, and GPT, Claude and Gemini all grew up on top of this design.

Try it — What does 'it' point to? Watch the words look at each other

Pick a word, and the colors show which words in the sentence it looks at, and how hard. Bright means a hard look; faded words barely get a glance.

Reading in order vs. looking all at once

When the way of reading changed, the way of calculating changed too.

Why the 2017 paper was titled 'Attention Is All You Need' — it threw out the read-in-order part completely.

1. Q1What's the biggest difference between the Transformer and the way before it?

   1. ① Words in a sentence look at each other at once
   2. ② It memorizes words faster
   3. ③ It searches the internet to answer
   4. ④ People type in rules by hand

2. Q2Why did 'looking at once' matter so much?

   1. ① It made sentences prettier
   2. ② All-at-once math is perfect for graphics cards
   3. ③ It used less memory
   4. ④ It was easier for people to read

3. Q3In 'it was too tired,' how does attention know 'it' means the animal?

   1. ① It's the first word
   2. ② A dictionary says so
   3. ③ Scores how well words fit
   4. ④ It picks at random

Once words could look at each other, AI started understanding whole sentences.

Why did just making it bigger give it new skills?

They didn't change yesterday's Transformer design at all — they just made it bigger, and in 2020 out came a model with 175 billion numbers. Its name is GPT-3.

The surprise wasn't the size, it was how it learned. With no retraining, you could tuck just two or three examples into your question and it would pick up the task right away.

That's why the paper was titled 'Language Models are Few-Shot Learners' — 'few-shot' means from just a few examples. From then on, 'what you tell it,' the prompt, started to really matter.

Try it — A feel for digits — how big did GPT get?

Move the slider to guess 'about 10 to the what?' first. Then reveal the answer to see the real number. The goal isn't to get it right — it's to feel how fast the digits jump.

How much GPT grew in 3 years

Measure them with the same ruler as bars, and the difference jumps out.

There was earlier research that worked out in advance how much better a model gets as you add numbers. GPT-3 was that math pushed all the way, for real.

1. Q1What made GPT-3 most different from the models before it?

   1. ① It searched the internet in real time
   2. ② Show it a few examples, and it did a new task
   3. ③ People put in the rules one by one
   4. ④ It switched its own power on and off

2. Q2About how many numbers (parameters) does GPT-3 have?

   1. ① About 170 million
   2. ② About 1.7 billion
   3. ③ About 175 billion
   4. ④ About 17.5 billion

3. Q3What do you call what you give the model, examples tucked in and all?

   1. ① An update
   2. ② A download
   3. ③ A refresh
   4. ④ A prompt

Same design, just more size and more reading, and it learned to copy from a few examples.

A chat window, not the model, changed the world?

On November 30, 2022, a model from the GPT family you met yesterday was put inside a chat window where you could talk back and forth and opened up to everyone. That was ChatGPT.

Nobody invented a brand-new model. The big thing was polishing it using which answers people liked better, so it actually understood what you asked it to do.

So what changed that day wasn't the technology, it was who got to use it. Something only people who knew how to add examples could use became something everyone used.

Try it — 2020 vs. 2022: what changed?

Pick an item and switch back and forth between 2020 and 2022. If you can see what changed besides the model's size, you've got it.

How a big model became 'something anyone uses'

Three layers of work stacked up to make November 2022.

Nov 30, 2022: the day a model from the same family showed up with a chat window — that's when people first got their own hands on AI.

1. Q1What's the biggest reason ChatGPT surprised the world?

   1. ① It invented a whole new design
   2. ② It was faster than the internet
   3. ③ Anyone could use it just by talking to it
   4. ④ It grew to 1 trillion numbers

2. Q2What did polishing with human feedback show?

   1. ① The bigger model always won
   2. ② Answers always got longer
   3. ③ Results were the same at any size
   4. ④ People preferred a 1.3B model over a 175B one

3. Q3What year did ChatGPT first come out?

   1. ① 2022
   2. ② 2020
   3. ③ 2017
   4. ④ 2025

Not a bigger model, but a window you could talk to, made AI something for everyone.

Why write out the work instead of answering right away?

The secret to AI suddenly getting good at hard problems wasn't getting bigger — it was making it go through the steps before answering.

At first, it worked when a person told it 'think step by step,' but later came models trained to write out their work on their own first. These are called reasoning models.

But there's a price. The longer it thinks, the slower the answer and the more text it writes. So for easy questions, it doesn't bother.

Try it — Put the steps of the work in order

Tap the mixed-up pieces in the order you think is right. When you've picked them all, it'll grade you. The point is to feel for yourself that the answer only comes out right if the order is right.

Answering right away vs. thinking first

Same model family, but just changing the order it answers in changes the results.

In 2022, a person had to tell it 'think step by step'; in 2024, models came out that do it without being told.

1. Q1What makes a reasoning model different from earlier models?

   1. ① It memorizes whole answers
   2. ② It makes its answers shorter
   3. ③ It always searches the internet
   4. ④ It works through the steps before answering

2. Q2What happened on the same math competition problems?

   1. ① It went up from 12% to 74%
   2. ② It dropped from 74% to 12%
   3. ③ Both got 100%
   4. ④ There was no difference

3. Q3What does thinking first cost?

   1. ① Accuracy
   2. ② Time and words written
   3. ③ Internet speed
   4. ④ Screen brightness

Once it wrote out its work instead of answering right away, it started getting right what it used to get wrong.

Is grabbing a calculator when you don't know the smarter move?

AI's main job is adding the next word, so it's weak at things you need to know exactly, like big multiplications or today's weather. So people hooked up tools next to it: a calculator, search, running code.

The important part isn't the tools, it's choosing for itself when to grab one. Since 2023, models can say 'I need a calculator now' in a set format, get the result back and finish the answer.

So a good AI isn't one that knows everything. It's one that notices it doesn't know and grabs a tool — the same call you make when you use a calculator on a test.

Try it — Answer directly, or grab a tool?

Here are six questions. Pick what an AI should do. Each pick gets graded right away, and once you've done them all, you'll see the rule.

Four steps to grabbing a tool

An AI that uses tools goes through these steps inside.

Starting in 2023, models began saying 'which tool to call, with what values' in a set format — and now that's a basic feature of almost every model.

1. Q1What does it mean for an AI to 'use tools'?

   1. ① It memorizes whole answers
   2. ② It calls a calculator, search or code on its own
   3. ③ It switches to a bigger model
   4. ④ It makes its answers shorter

2. Q2Which of these questions is better handled with a tool?

   1. ① What's 'cat' in Spanish?
   2. ② What's the opposite of 'big'?
   3. ③ What's 847,293 times 6,184?
   4. ④ What's 3 plus 4?

3. Q3Who decides whether to call a tool?

   1. ① The power button
   2. ② The user, every time
   3. ③ The internet line
   4. ④ The model itself

Noticing you don't know and grabbing a tool is a kind of smart, too.

How do photos and sounds get to stand next to words?

The secret to AI seeing photos and hearing sounds wasn't giving it eyes and ears. Photos and sounds got cut into small pieces too, and lined up in the same row as word pieces.

Text gets cut into word pieces, photos into square tiles on a grid, and sound into very short slices of time. Once they're cut, all three are just 'pieces in a row.'

That means there's no need to build a new design. You can feed them straight into the Transformer, which lets pieces look at each other. Because it handles several senses at once, it's called multimodal.

Try it — Split text, photos and sound into pieces

Tap the buttons to switch what goes in. If you can see that whatever you put in ends up as pieces standing in a row, you've got it. Orange marks the pieces to watch right now.

Whatever goes in takes the same road

Three kinds of input, but only one road.

In 2023, the ability to read photos opened up widely; in 2024 came a model that learned text, photos and sound in one design, and its spoken replies got as fast as 0.32 seconds on average.

1. Q1What actually happens when you put a photo into an AI?

   1. ① The photo gets translated into text first
   2. ② It's turned black and white
   3. ③ It's cut into a grid and the pieces lined up
   4. ④ It's made twice as big

2. Q2Why can text, photos and sound all go through one design?

   1. ① The model was given eyes and ears
   2. ② Photos were text all along
   3. ③ Sound has no meaning
   4. ④ All three become pieces in the same row

3. Q3How does sound get split up?

   1. ① Short time slices
   2. ② One letter at a time
   3. ③ By color
   4. ④ Sorted by length

Chop photos and sounds into pieces and line them up, and they can stand right where words do.

Why are there so many AIs you can use now?

At the end of 2022, you could count the really usable chat AIs on one hand. Now dozens of places — in the US, Europe, China, even South Korea — each put out their own models.

The biggest change was open models. A model's whole bundle of numbers got released, so anyone could download it and run it on their own machine.

So now it's an age of picking which one to use, not just using the one. Price, speed and strengths all differ, and when one place releases a new feature, similar ones pop up all over within a few months.

Try it — End of 2022 vs. now: what changed?

Pick an item and switch back and forth between 'End of 2022' and 'Now.' The key isn't the numbers — it's how 'being able to choose' changed.

Size of models anyone could download

Let's see in numbers how fast open models grew.

Once the numbers were released, schools and small research teams started modifying models themselves — and that created the race we see today.

1. Q1What's the biggest difference between the end of 2022 and now?

   1. ① Prices to use it went way up
   2. ② All the models merged into one
   3. ③ Nobody makes them anymore
   4. ④ Way more places make models now

2. Q2What does 'open model' mean?

   1. ① Anyone can download its numbers and run it
   2. ② A model with no ads
   3. ③ A model that always gives the right answer
   4. ④ A model that searches without the internet

3. Q3When more places make models, what happens for users?

   1. ① Answers disappear
   2. ② You get to choose
   3. ③ The power goes out
   4. ④ The text gets bigger

As the places making it went from a few to dozens, AI went from something you use to something you choose.