Part 5 Review
From one dot to the phone in your pocket · one layer deeper · DAY 46-55
AIMade by AIThis part review was written by AI (Claude) from that part's lessons and records, and published after automated checks. The owner doesn't read every item.How it's made
Part 5 spent ten days on what the inside of an AI looks like. It started with what one dot does, then turned meaning into numbers, looked things up and laid them open, tuned the model, and ended with the metal and the electricity that run it all.
No quizzes and no experiments here. It's a read-and-move-on piece. If a day feels fuzzy, jump straight to that day's lesson.
DAY 46 · Neural networks
One dot can only multiply and add
A neural network is dots passing numbers around, stacked in layers. One dot multiplies each incoming number by a weight and adds them up, then just checks whether the total clears a threshold. Clear it and the signal goes to the next layer; miss it and it stops there.
The first layer only catches small pieces, like lines and dots. The next layer builds eyes and ears from those pieces, and the last layer says cat — that's what deep means. Without the threshold, stacking any number of layers collapses back into one multiply-and-add.
DAY 47 · Attention
Attention is a share you split up
Attention is the math that decides how hard a word stares at the other words. The raw scores aren't used as they are; they become shares adding up to 1 and get split around. So staring hard at one word automatically weakens all the rest.
Who "it" refers to gets settled when the staring piles onto the word that fits. The big shift was making a sentence look at itself all at once instead of reading front to back in one line. That 2017 method became the base of almost every model in use today.
DAY 48 · Meaning as numbers
Meaning gets turned into a place
A computer can't read letters, so each word is stored as a bundle of several numbers. Nobody hands it those numbers — it works them out by itself while reading an enormous amount of text. There is one rule: words that show up in similar company get similar numbers.
Once meaning is a place, near and far can be measured as numbers. By letters, apple and orange have nothing in common; by place they sit side by side. That's the base for finding similar things, which is the next day.
DAY 49 · Finding similar things
It measures meaning, not letters
Sentences, like words, can become number bundles. Then a distance between two of them falls out. Even with no letters in common, same meaning means a close place. And nearly identical letters with the opposite meaning move apart.
"That was fun" and "that was no fun" share almost every letter but mean the opposite. Measure by letter overlap alone and you would score those two as very close, which is wrong. That's why finding text moved from letter matching to places of meaning.
DAY 50 · Answering with the book open
Look it up and read it, instead of reciting
Answering from memory alone makes inventing a plausible answer easy. So the question becomes a number bundle and the text pieces sitting in close places get pulled out first. Then it reads those beside the question and answers.
You don't have to retrain the model — swap out only the text you lay open and the answers follow. It can show where the words came from, which makes checking easy. But if the text it found is wrong, the answer is wrong with it. Finding is half the job.
DAY 51 · Teaching it extra
Nudge a grown model instead of raising a new one
Teaching one more job on top of a base model that already did the big round of learning is called fine-tuning. The numbers inside stay almost untouched; only a tiny slice changes. A method called LoRA cut the numbers it touches down to one ten-thousandth.
Teaching it extra is strong at tone and format. If you want new facts in there, answering with the book open is the right fit. The big round takes months; the tuning step is far shorter and cheaper by comparison.
DAY 52 · People handing out scores
Not the right answer, just which one is better
You get two answers to one question, have a person pick the better one, and repeat tens of thousands of times. What you collect isn't right-and-wrong, it's what people prefer. Scoring every answer by hand would never end, so the picks are used to build a scoring model first.
From then on that scoring model hands out the points, and the AI gets tuned toward them. In a study released in 2022, people preferred this tuned small model's answers over those of a model more than 130 times its size. It's a rare case where tuning mattered more than size.
DAY 53 · GPUs
The shape of the math was the same all along
A graphics card (GPU) is a part built to do easy math a huge number of times at once. Painting millions of dots on a screen at the same time needs thousands of simple workers, not a few clever ones. AI training is the same: not one hard problem, but trillions of easy multiplications at once.
One card of the kind used for training holds about 80 billion tiny switches. That single card does roughly 1,000 trillion calculations a second. Even that wasn't enough: Llama 3 ran 16,000 cards as one block for 54 days.
DAY 54 · Data centers and electricity
One is tiny, the pile is not
In a measurement released in 2025, one text question came in at a median of 0.24 watt-hours. That's about a TV left on for 9 seconds, so one question alone is nothing to worry about. The catch is that questions like that keep piling up, nonstop.
Data centers worldwide used 415 terawatt-hours of electricity in 2024, about 1.5% of all electricity. The International Energy Agency expects 945 terawatt-hours by 2030, more than double. 1.5% looks small, but things that double in six years are not common.
DAY 55 · AI inside my phone
It splits the work into pieces
Today's phones carry a small model of about 3 billion numbers. Even with no internet it can summarize, polish text and tidy up notifications. The model on the phone and the one on the server differ in size by tens to hundreds of times.
The phone tries first and sends out only the pieces that are too heavy for it. Finishing inside means faster, less electricity, and your words never leaving. Long text and hard questions still have to travel to that building from before.
Part 5 in one line
Tie the ten days together and you get this - there's no magic inside, just simple math piled up enormously. Multiply and add (DAY 46), decide where to look (DAY 47), turn meaning into a place (DAY 48-49), look it up and lay it open (DAY 50), tune it (DAY 51-52), and then the metal and the electricity and the phone that run it all (DAY 53-55).
Part 6 looks ahead. Is an AI that matches people really coming, and why do people argue over those words?
Learn more
- Deep learning (Nature, 2015) — a short summary written by the three people who built deep learningnature.com/articles/nature14539
- Learning representations by back-propagating errors (1986) — the paper that made fixing weights by the size of the error widely knownnature.com/articles/323533a0
- McCulloch & Pitts, A logical calculus... (1943) — the first paper to treat a nerve cell as a computing elementlink.springer.com/article/10.1007/BF02478259
- Neural Machine Translation by Jointly Learning to Align and Translate (2014) — the translation study where attention was first usedarxiv.org/abs/1409.0473
- Attention Is All You Need (2017) — the paper that dropped reading in order and kept only attentionarxiv.org/abs/1706.03762
- Efficient Estimation of Word Representations (2013) — the study that spread the idea of turning words into number bundlesarxiv.org/abs/1301.3781
- Sentence-BERT (2019) — a way to build sentence bundles that are good for measuring distancearxiv.org/abs/1908.10084
- Dense Passage Retrieval (2020) — finding text by places of meaning instead of letter matchingarxiv.org/abs/2004.04906
- Retrieval-Augmented Generation (2020) — the study that had a model answer with found text laid open beside itarxiv.org/abs/2005.11401
- LoRA (2021) — the method that cut the numbers you touch down to one ten-thousandtharxiv.org/abs/2106.09685
- Training language models to follow instructions with human feedback (2022) — tuning a model with records of what people pickedarxiv.org/abs/2203.02155
- NVIDIA Hopper Architecture In-Depth (2022) — the makers' own walkthrough of the card used for trainingdeveloper.nvidia.com/blog/nvidia-hopper-architectur…
- The Llama 3 Herd of Models (2024) — the record of a 54-day run on 16,000 cardsarxiv.org/abs/2407.21783
- Measuring the environmental impact of AI inference (Google, 2025) — a company-reported measurement of the electricity one question takescloud.google.com/blog/products/infrastructure/measu…
- Energy and AI - Executive summary (IEA, 2025) — an official summary putting data center electricity in national termsiea.org/reports/energy-and-ai/executive-summary
- Introducing Apple's On-Device and Server Foundation Models (2024) — an official post on splitting work between a phone model and a server modelmachinelearning.apple.com/research/introducing-appl…