AI / LLM Basics Interview Questions
What is Model Distillation?
Model distillation is a technique for training a smaller "student" model to mimic the behavior of a larger "teacher" model, transferring much of the larger model's capability into a more efficient package.
- The student model is trained to match the teacher's output probabilities, not just its final answers
- Produces a smaller model that runs faster and cheaper than the original
- Usually results in some capability trade-off compared to the full-sized teacher model, though a well-distilled model can retain much of its usefulness
Distillation is one of several complementary approaches, alongside quantization, for making a powerful but expensive model practical to deploy at scale.
What does the student model in distillation try to mimic?
What does distillation typically produce?
More Related questions...
