Part 1 Review
The same ten days, plus the why · one layer deeper · DAY 01–10
The facts are exactly the same as in the Beginner review. I just added one more layer: why it works that way.
When a new term shows up, I'll explain it in one sentence the first time. Read in order, and the ten days will link up into one line.
DAY 01 · Guessing the next word
Pick a piece, add it, pick again
AI doesn't make up a whole sentence at once. It picks just the next piece, adds it, then picks again. Repeat that pick hundreds of times, and you get the long answer we read. So AI isn't fetching an answer stored somewhere — it builds a new one every time.
Because that's exactly the job it got in training — cover part of a sentence and guess the next piece. A machine that learned to guess well works the same way when it answers. That's also why answers seem to trickle out bit by bit. That really is how they're being made.
DAY 02 · Tokens
That piece has a name: token
The piece AI picks has a name. It's called a token. A token isn't a letter or a word — common chunks stay whole, and unfamiliar ones get chopped up small. To you, 'What's for lunch today?' is four words. To an AI, it's about six tokens.
The cutting rules are called a tokenizer. Each model has its own, so token counts for one sentence vary. More tokens means more math to do — so it gets slower and pricier. Languages like Korean often need more tokens than English to say the same thing.
DAY 03 · Probability and temperature
The probability table and the temperature knob
To pick the next piece, AI scores every candidate, then holds a raffle where higher scores get more tickets. Temperature is the knob that sets how adventurous that raffle gets. Turn it down and #1 wins every time; turn it up and lower candidates pop up more often.
Same question, different answer each time? That's not a mood — it's the raffle. For jobs with one right answer (math, summaries, code), turn it down; when you want lots of ideas, turn it up. Turning it up doesn't make it smarter — just more varied, and more likely to say something off the wall.
DAY 04 · Training
Training is tiny fixes, repeated
AI training isn't a sudden 'aha' — it's 'tiny fixes,' repeated. Hide part of a sentence, have it guess the next piece, then nudge the numbers inside by how much it missed. Repeat that an enormous number of times, and grammar and common sense soak into those numbers.
It's close to giving every miss an 'off by this much' score, then shifting a little in that direction. No big fixes all at once. Small ones — but a huge number of times. That's why training takes huge amounts of time and electricity — and once it's done, the numbers are locked.
DAY 05 · Parameters
Parameters: how many numbers get tuned
How many numbers training tunes — that's the parameter count. The more there are, the bigger the container for patterns. Today's big models run from tens of billions to around 1 trillion — not a number you could ever count.
Counting one per second, 100 billion would take you more than 3,000 years. The human brain has about 100 trillion synapses, so models still fall short of that. But bigger doesn't always mean smarter — what it trained on, and how much, has to come with it.
DAY 06 · Hallucination
Confidently making things up — hallucination
When AI calmly makes up facts that don't exist, it's called hallucination. It isn't trying to lie. It comes naturally with a setup that picks a plausible next piece. AI has no separate built-in way to flag 'I don't know this.'
It slips most on things rare in its training data, very recent events, and specifics: numbers, dates, names. It sounds the same whether it knows or not. Looking confident doesn't make it true. The habit of asking for sources and opening them yourself is the cheapest, surest defense.
DAY 07 · Memory limits
Not remembering — rereading
AI doesn't save your chat. Every time, it rereads the whole conversation so far, then answers. There's a limit to how much it can read at once. That limit is called the context length. As the chat gets longer, the early part gets pushed out of the window.
Sometimes 'I already told you!' doesn't work. What you said may already be out the window. Even with room to spare, there's a catch — studies found the middle tends to count less than the start or end. For really important info, the surest move is to point it out again later in the chat.
DAY 08 · Prompts
New question, new probability table
A prompt is the whole input you hand to AI. Change the words, and the probability table for the next piece changes too — so the answer changes a lot. You're not changing the AI — you're changing the question.
Give it a role, spell out the conditions, name the format you want — these three work best. 'A 100-word notice for middle schoolers, with one example' gets way better answers than 'Write something good.' It's not a trick. You're just saying what you want more clearly.
DAY 09 · Big models, small models
Size means parameter count
A 'big' model just means it has lots of parameters — the ones from DAY 05. Big models know a lot and can handle hard problems, but they're slow, pricey and only run on servers. Small models fit on your own device, so they're fast and work even without internet.
Quick, frequent, light jobs go small; jobs that need long thinking and lots of reading go big. You can also teach a small model, using a big model's answers as its textbook. Then the small model punches above its weight — but it can't fully replace the big one.
DAY 10 · Open and closed models
Where the core lives
A closed model's core stays on the company's servers. We can only talk to it through the front desk. With an open model, you can download that core — the weights — in full and run it on your own computer. You ask questions the same way — the difference is where the model lives.
'Open' isn't one thing. Split it into four: weights, code to run it, training data, the right to change it. Most open up the weights and code, but not the training data. So don't stop at the word 'open' — ask which pieces are actually open.
Put it all together and you get this
DAY 01's 'picking the next piece' is where everything starts. Give the piece a name and it's a token (DAY 02); the way it picks is probability and temperature (DAY 03).
What made those probabilities is training (DAY 04), and how many numbers training adjusts is the parameter count (DAY 05). Hallucination (DAY 06) and memory limits (DAY 07) simply come with this setup — they aren't glitches.
Prompts (DAY 08) are the knob we get to hold, and size (DAY 09) and openness (DAY 10) were about where and how to put the same thing.
Learn more
- Attention Is All You Need (2017) — the Transformerarxiv.org/abs/1706.03762
- Neural Machine Translation of Rare Words with Subword Units (2015) — the BPE tokenizerarxiv.org/abs/1508.07909
- Learning representations by back-propagating errors (1986) — backpropagationnature.com/articles/323533a0
- Language Models are Few-Shot Learners (2020) — GPT-3, 175 billion parametersarxiv.org/abs/2005.14165
- Training Compute-Optimal Large Language Models (2022) — Chinchillaarxiv.org/abs/2203.15556
- A Survey on Hallucination in Large Language Models (2023)arxiv.org/abs/2311.05232
- Lost in the Middle (2023) — the middle of a long input gets used lessarxiv.org/abs/2307.03172
- Pre-train, Prompt, and Predict (2021) — a survey of promptingarxiv.org/abs/2107.13586
- Distilling the Knowledge in a Neural Network (2015) — distillationarxiv.org/abs/1503.02531
- Llama 2 release (2023) — open weights and the license debateen.wikipedia.org/wiki/Llama_(language_model)
- Mistral 7B (2023) — Apache 2.0mistral.ai/news/announcing-mistral-7b