Token: A token is the unit of text a language model reads and writes, usually a word or a piece of a word. Models measure input limits, output limits, and pricing in tokens; as a rough rule, 1,000 tokens is about 750 words of English.
Language models do not see letters or whole words. Before your prompt reaches the model, it is chopped into tokens by a fixed vocabulary. Common words like 'the' or 'invoice' are usually one token. Longer or rarer words get split: 'reconciliation' might become three pieces. Numbers, code, and non-English text often cost more tokens per word.
Every limit you bump into is measured in tokens. The context window is a token count. API pricing is per million tokens in and out. The maximum length of a single response is a token count. When a tool says a document is too long, it means too many tokens.
Tokens also explain some odd behavior. A model can struggle to count letters in a word or reverse a string because it never saw the letters, only the tokens. And two prompts that look similar in length can cost very different amounts if one is full of numbers, symbols, or unusual formatting.
Example at work
An operations manager wants to summarize a year of weekly status reports, about 200,000 words. At roughly 750 words per 1,000 tokens that is around 270,000 tokens, more than many models accept at once. She splits the reports by quarter, summarizes each batch, then asks the model to combine the four summaries.
Why it matters
Knowing about tokens helps you plan work that fits the tool. You will know why a huge spreadsheet fails to paste, why a long chat starts forgetting early instructions, and, if your team uses an API, why the monthly bill is what it is.
Related terms
- Context windowThe context window is the maximum amount of text, measured in tokens, that a language model can consider at one time. It includes your prompt, any pasted documents, the conversation so far, and the model's reply.
- Large language model (LLM)A large language model (LLM) is a neural network trained on enormous amounts of text to predict the next word, then tuned to follow instructions and hold a conversation. LLMs power ChatGPT, Claude, Gemini, and Microsoft Copilot.
- API (application programming interface)An API (application programming interface) is a way for one piece of software to talk to another. An AI API lets your own tools, spreadsheets, or automations send a prompt to a model and receive the response, without a person typing in a chat window.
- Rate limitA rate limit is a cap on how much you can use an AI service in a given period, such as messages per hour on a chat plan or requests and tokens per minute on an API, set to protect the provider's capacity and to tier pricing.
- InferenceInference is the act of running a trained AI model to produce an output, such as answering a prompt or classifying an image. Training happens once; inference happens every time anyone uses the model, and it is what you pay for as a user.