Retrieval-augmented generation (RAG): Retrieval-augmented generation (RAG) is a technique in which an AI system first searches a set of documents for relevant passages and then gives those passages to a language model to answer from. It lets an assistant use your own, current information.
A language model only knows what it learned in training, so it cannot answer questions about your policies, your contracts, or last week's meeting. RAG solves that without retraining. When you ask a question, the system retrieves the most relevant chunks from your documents and pastes them into the prompt along with your question. The model then writes an answer grounded in those chunks.
Under the hood, documents are split into passages and converted into embeddings, stored in a vector database, and matched by semantic search. But you meet RAG every time you use a 'chat with your documents' feature, a company knowledge assistant, or a search tool that shows citations under its answer.
RAG reduces hallucination but does not eliminate it. If the retrieval step misses the right passage, the model answers from thin air or from the wrong document. Quality depends on clean, well-organized source material and on the model being told to say when the documents do not contain the answer. Always click through to the cited source when it matters.
Example at work
An HR team builds an internal assistant over the employee handbook, benefits guides, and policy memos. When an employee asks how parental leave works for part-time staff, the assistant retrieves the two relevant sections and answers with a link to each. The HR team reviews questions the assistant could not answer and updates the documents.
Why it matters
RAG is how most organizations will actually put AI on top of their own knowledge. Knowing the term helps you evaluate vendor claims, understand why an assistant gives good answers on some questions and bad ones on others, and see that the fix is usually better documents, not a better model.
Related terms
- EmbeddingAn embedding is a list of numbers that represents the meaning of a piece of text, an image, or other content, so that items with similar meaning end up with similar numbers. Embeddings are what let AI systems search by meaning instead of exact words.
- Vector databaseA vector database stores embeddings, the numeric representations of text, images, or other content, and finds the items closest in meaning to a query quickly. It is the storage layer behind semantic search and most document-aware AI assistants.
- Semantic searchSemantic search finds results based on the meaning of a query rather than exact keyword matches, so a search for 'unpaid invoices' also returns documents about 'overdue bills.' It uses embeddings to compare meaning.
- GroundingGrounding means tying an AI model's answer to specific, verifiable sources, such as documents you provide, a database, or live search results, instead of relying on what the model remembers from training.
- HallucinationA hallucination is when an AI model produces information that is false or fabricated but presents it confidently, such as an invented statistic, a citation to a paper that does not exist, or a made-up product feature.