Storing a model's numbers with less precision so it takes up less memory.
An AI model is made of billions of numbers (weights), and each one is usually stored using about 16 bits. Quantization rounds those numbers off so they fit in 8 or even 4 bits. Going from 16 bits to 4 bits cuts the memory the model needs to about a quarter.
That's how models built for big data centers can run on a home computer or even a phone. Squeeze too hard, though, and answer quality can drop, so researchers keep working on smarter ways to choose what to shrink and by how much. Some models are shrunk after training; others are trained with the shrinking in mind.
Think of it like this
Think of writing pi as 3.14 instead of 3.14159265. It takes far less space in your notebook, and most everyday sums come out almost the same. Round it all the way down to 3, though, and your answers start to go noticeably wrong.
For example
Google released a 4-bit version of its Gemma 3 27B model that shrank it from about 54 GB to about 14 GB.
Records about this
Source-checked moments from AI history.
Quantization-aware training became a standard step in open-weight releases
Standardized quantization as a post-processing step
Made it possible to run large models on consumer-grade GPU servers