Retrieval-augmented generation

A typical RAG pipeline has three steps. Documents are split into chunks and converted into embeddings stored in a vector database or search index. When a user asks a question, the system retrieves the chunks most similar to it. Those chunks are inserted into the prompt, and the model generates an answer grounded in them, often with citations. The term comes from a 2020 paper by Lewis et al.

RAG is how most "chat with your documents" products, AI help desks and enterprise search assistants work. It lets a general model answer questions about private or recent information without retraining. When comparing such tools, ask how they chunk and index content, whether answers cite sources, how permissions are enforced and how often the index refreshes.

Example: an HR assistant answers "How many days of parental leave do we offer?" by retrieving the relevant section of the company handbook and quoting it.

Limits: RAG is only as good as its retrieval. If the right passage is not found, or the documents are outdated or contradictory, the model can still give a wrong answer. It reduces hallucination but does not eliminate it.