Retrieval-augmented generation (RAG)

Retrieval-augmented generation, usually shortened to RAG, is a technique that combines a language model with a retrieval step over a body of documents. When a question is asked, the system first searches a knowledge source for relevant material, then supplies that material to the model as context so its answer is grounded in specific, current documents rather than relying solely on what was captured during training.

RAG addresses two persistent limitations of language models. It lets a model draw on private, up-to-date or specialised information it was never trained on — an organisation’s own documents, for example — without retraining the model. And by grounding responses in retrieved sources, it reduces the tendency to produce fluent but unsupported answers, and makes it possible to cite where an answer came from, which matters where responses must be checkable.

RAG is often the right first step for putting an organisation’s knowledge to work with AI, because it keeps the authoritative content in a controlled store rather than baking it into model weights, and content can be updated or removed without touching the model. It carries its own considerations: the quality of answers depends heavily on the quality of retrieval, access controls must be enforced so the retrieval step cannot surface documents a user should not see, and retrieved content is a vector for indirect prompt injection if it is not treated as untrusted.