Model distillation: Model distillation is a training technique in which a smaller 'student' model learns to imitate the outputs of a larger 'teacher' model, producing a model that is cheaper and faster to run while keeping much of the teacher's capability.
Training a great model is expensive; running it is expensive too. Distillation is a way to get most of the benefit at a fraction of the running cost. The large teacher model generates answers, explanations, or step-by-step reasoning across a wide range of prompts. The small student model is then trained to reproduce those outputs. Because it learns from the teacher's polished answers rather than raw internet text, it ends up far stronger than a small model trained from scratch.
This is how many of the fast, cheap models in everyday products are made. Vendors typically train a flagship and then distill it into lighter tiers for high-volume use. Google has described its Gemini Flash models this way, and DeepSeek released small open models distilled from its R1 reasoning model in 2025. The student inherits the teacher's style and much of its knowledge, but not all of it, and it usually falls short on the hardest problems.
Distillation also raises a business question. Most major vendors' terms prohibit using their model outputs to train a competing model, and public disputes over exactly that have already happened. If your company is building its own model, check the terms of any AI service whose outputs feed into it.
Example at work
A customer success team wants an AI assistant inside their help desk that replies in under a second. The frontier model they piloted was accurate but slow and pricey at their volume. Their vendor offers a distilled version of the same model, which answers common questions just as well and escalates the tricky ones to the full model.
Why it matters
Distillation explains why the 'mini' or 'flash' version of a model exists, why it is so much cheaper, and where it will fall short. It is also worth knowing before your organization trains anything on another vendor's outputs, since that practice sits in contested legal territory.
Related terms
- Small language model (SLM)A small language model (SLM) is a language model with far fewer parameters than a frontier model, small enough to run on a laptop, phone, or single server. It trades some capability for speed, low cost, and the option to run privately on your own hardware.
- Fine-tuningFine-tuning is extra training that adjusts an existing AI model's weights using a smaller, targeted set of examples so it performs better on a specific task, style, or domain. It changes the model itself, unlike prompting, which only changes the instructions.
- Large language model (LLM)A large language model (LLM) is a neural network trained on enormous amounts of text to predict the next word, then tuned to follow instructions and hold a conversation. LLMs power ChatGPT, Claude, Gemini, and Microsoft Copilot.
- Synthetic dataSynthetic data is artificially generated data, often produced by an AI model, that mimics the statistical patterns of real data and is used to train, test, or demonstrate systems without exposing real records.
- Training dataTraining data is the collection of examples an AI model learns from, such as web pages, books, code, images, or labeled records. What is in that data, and what is missing, shapes everything the model knows, how it writes, and which mistakes it makes.