AI WordsBasicsIntermediateone layer deeper

Parameter, weights · parameter count

One of the many numbers inside a model that get tuned during training.

Parameters are the numbers inside a model. The weights on the connections in a neural network are the main kind. Training is really just adjusting these numbers bit by bit until they combine to give good answers.

People often describe a model's size by its parameter count. GPT-3, from 2020, had about 175 billion; small models that run on phones have a few billion. More parameters can hold more patterns, but more isn't automatically smarter: what the model learned, and how well, matters too.

Think of it like this

Picture a sound mixing desk covered in knobs. Each knob is a parameter, and training is turning them little by little until it sounds right. An AI just has billions of knobs.

For example

In "a 7B model", the B stands for billion: about 7 billion parameters.