Small language model (SLM): A small language model (SLM) is a language model with far fewer parameters than a frontier model, small enough to run on a laptop, phone, or single server. It trades some capability for speed, low cost, and the option to run privately on your own hardware.
'Small' is relative. Frontier models have hundreds of billions of parameters or more and run in data centers. Small models have a few billion or fewer, and a well-made one runs on a modern laptop with no internet connection. Microsoft's Phi family, Google's Gemma, and the smaller Llama and Mistral releases are common examples, and phone makers now ship small models inside their devices for tasks like summarizing notifications.
What you give up is breadth. A small model knows less, reasons less reliably, and is more prone to confident errors on hard questions. What you gain is speed, cost, and control. It can be fine-tuned on your own data, it can process documents without those documents leaving your building, and it costs a fraction of a frontier model per call. For narrow, repetitive tasks such as sorting support tickets or pulling fields from invoices, a tuned small model often matches a large one.
Many production systems now use both: a small model handles the high-volume, easy cases and routes anything unusual to a larger model. If you are choosing a tool, ask which model it uses and whether it runs on the vendor's servers, your own, or the device in your hand.
Example at work
A healthcare administrator needs to tag thousands of incoming patient messages by department. Sending them to a consumer cloud AI would expose PHI, so IT runs a small model on an internal server. It sorts messages well enough, the few it marks 'unsure' go to a person, and nothing leaves the network.
Why it matters
Small models are how AI gets into places where cost, speed, or privacy rule out a frontier model. Understanding the tradeoff helps you spot when a cheaper, private option is good enough and when the task really does need the big model.
Related terms
- Large language model (LLM)A large language model (LLM) is a neural network trained on enormous amounts of text to predict the next word, then tuned to follow instructions and hold a conversation. LLMs power ChatGPT, Claude, Gemini, and Microsoft Copilot.
- ParametersParameters are the adjustable numbers inside an AI model, mainly its weights, that are learned during training. The parameter count, from a few billion to over a trillion, is the standard shorthand for a model's size and rough capability.
- QuantizationQuantization is a compression technique that stores a model's numerical weights with less precision, for example 8 or 4 bits instead of 16, so the model needs less memory and runs faster, usually with only a small loss in quality.
- Model distillationModel distillation is a training technique in which a smaller 'student' model learns to imitate the outputs of a larger 'teacher' model, producing a model that is cheaper and faster to run while keeping much of the teacher's capability.
- Open-weights modelAn open-weights model is an AI model whose trained parameters (weights) are published for anyone to download, run, and modify, usually under a license that sets limits on commercial use or redistribution.