Just how big is this number? — Getting a feel for parameters
A bigger number doesn't always mean smarter.
1 · Think first
Parameters are how many weight numbers keep getting adjusted during training. The more there are, the bigger the container for patterns.
Today's big models have tens of billions to 1 trillion parameters. That's still short of the number of synapses in a human brain (about 100 trillion).
More parameters doesn't automatically mean smarter — what it learned, and how, has to come along with it.
2 · Experiment · Number sense
Try it — the number-sense slider
Move the slider to guess how many digits it has (10 to what power?), then see how close you are to the real value.
Figure
In one picture
Put the three numbers side by side and you can feel how different their digit counts are.
3 · Quiz
Quick check
Three questions. Pick an answer and it's marked right away, with the reason why.
Today in one line
Today in one line
A bigger number
doesn't always mean smarter.
Learn more
- One of the first times a parameter count made big headlines was GPT-3, released in 2020 — Language Models are Few-Shot Learners