Inference
Quantization
Quantization represents model values with fewer bits or a smaller numeric format.
How it works
Reduced precision lowers memory traffic and can accelerate supported hardware kernels. Calibration and outlier handling determine how much accuracy is retained.
A concrete example
Illustrative: 1 billion weights occupy about 2 GB at 16 bits and 0.5 GB at 4 bits, excluding metadata.
Do not confuse it with Pruning
quantization lowers precision; pruning removes weights or structures.