Mixture of experts (MoE): A mixture of experts (MoE) is a model design that splits the network into many specialized sub-networks called experts and activates only a few of them for each token, giving the capacity of a huge model at the running cost of a much smaller one.
In a standard model, every parameter participates in every token it processes. That is like routing every customer question through every department. A mixture-of-experts model instead contains dozens or hundreds of expert blocks and a small router that picks, token by token, which handful to use. The whole model may hold hundreds of billions of parameters, but only a fraction are active on any given step, so it runs fast and cheap relative to its size.
The experts are not specialists in the human sense. No single expert 'knows accounting.' They are divisions of labor that emerge during training, often around patterns in syntax or token type rather than topics. The design is now common at the top of the field: Mistral's Mixtral series, DeepSeek's V3 and R1, and Meta's Llama 4 models are publicly documented MoE designs, and Google has said its Gemini 1.5 models use the approach.
The tradeoff is memory. Even though only a few experts run at a time, all of them must be loaded, so an MoE model needs a lot of hardware to host even when it is cheap to run per token. That is why MoE models are common in data centers and rare on laptops.
Example at work
A small-business owner comparing API pricing notices that one open model advertises a very large total size but bills like a much smaller one. The vendor's documentation explains it is a mixture-of-experts design where only a small portion of the model is active per token. He tests it on his invoice-parsing task and finds it as accurate as the pricier option.
Why it matters
MoE is why some models can be enormous and still fast and affordable. Knowing the term helps you read model announcements without being misled by parameter counts and understand why the same 'size' can mean very different costs.
Related terms
- ParametersParameters are the adjustable numbers inside an AI model, mainly its weights, that are learned during training. The parameter count, from a few billion to over a trillion, is the standard shorthand for a model's size and rough capability.
- Large language model (LLM)A large language model (LLM) is a neural network trained on enormous amounts of text to predict the next word, then tuned to follow instructions and hold a conversation. LLMs power ChatGPT, Claude, Gemini, and Microsoft Copilot.
- TransformerThe transformer is the neural network architecture behind nearly all modern language models. Its key idea, called attention, lets the model weigh every word in a passage against every other word, so it can track meaning across long stretches of text.
- InferenceInference is the act of running a trained AI model to produce an output, such as answering a prompt or classifying an image. Training happens once; inference happens every time anyone uses the model, and it is what you pay for as a user.
- Open-weights modelAn open-weights model is an AI model whose trained parameters (weights) are published for anyone to download, run, and modify, usually under a license that sets limits on commercial use or redistribution.