Long context: Long context describes an AI model's ability to take in and work with very large inputs, from a whole contract to an entire codebase, in a single request. It is set by the size of the model's context window, measured in tokens.
Every language model has a context window: the maximum amount of text it can hold at once, counted in tokens. Early chat models topped out at a few thousand tokens, roughly a few pages. Current frontier models advertise windows of hundreds of thousands of tokens, and some claim a million or more, which is enough for several novels, a year of meeting transcripts, or a mid-sized software project.
A bigger window changes what you can ask. Instead of summarizing a contract one section at a time, you drop in the whole thing and ask which clauses conflict. Instead of describing your data, you paste the export. For a large, constantly changing document library, retrieval still wins because it finds the relevant pages and sends only those; long context is for when you want the model to see everything at once.
Long context is not free and not perfect. You pay for every token you send, so a huge prompt costs more and responds more slowly. Models also tend to recall material near the beginning and end of a long input better than material buried in the middle, a pattern documented in a 2023 paper by Nelson Liu and colleagues titled "Lost in the Middle". Vendors have improved on this, but the practical rule holds: put the most important material first, ask specific questions, and check that the answer actually cites the section you care about.
Example at work
A paralegal pastes a 180-page lease and the landlord's 40-page amendment into a single request and asks the assistant to list every clause where the amendment changes a deadline, quoting both versions side by side. She spot-checks three of the pairs against the original PDFs before sending the table to the attorney.
Why it matters
Long context is what makes AI useful for real documents rather than snippets. Knowing your tool's window size, and its weak spot in the middle of long inputs, tells you when to paste everything and when to break the job into pieces.
Related terms
- Context windowThe context window is the maximum amount of text, measured in tokens, that a language model can consider at one time. It includes your prompt, any pasted documents, the conversation so far, and the model's reply.
- TokenA token is the unit of text a language model reads and writes, usually a word or a piece of a word. Models measure input limits, output limits, and pricing in tokens; as a rough rule, 1,000 tokens is about 750 words of English.
- Retrieval-augmented generation (RAG)Retrieval-augmented generation (RAG) is a technique in which an AI system first searches a set of documents for relevant passages and then gives those passages to a language model to answer from. It lets an assistant use your own, current information.
- Context engineeringContext engineering is the practice of choosing what an AI model sees when it works on a task: instructions, documents, examples, tool results, and history, and how they are arranged. It extends prompt engineering from wording to the whole input.
- AI memoryAI memory is a feature that lets an AI assistant retain facts across separate conversations, such as your role, preferences, and ongoing projects, so you do not have to repeat them. It differs from the context window, which lasts only for one conversation.