Training
Knowledge distillation
Knowledge distillation trains a smaller student to match signals from a larger teacher.
How it works
Soft teacher probabilities expose relationships that hard labels omit. The student trades some capacity for lower inference cost.
A concrete example
Illustrative: a 1.3B student matches logits from a 7B teacher on 2 million sequences.
Do not confuse it with Quantization
distillation trains another model; quantization stores or computes the same model at lower precision.