Retrieval-augmented generation (RAG)

In a RAG system, documents are split into chunks and indexed, usually with embeddings. When a question arrives, the most relevant chunks are retrieved and passed to the model with instructions to answer from them, ideally with citations. A general model can then answer accurately about your products, procedures and standards without retraining, and you update its knowledge by updating documents. The approach was described in a 2020 research paper and has become standard for knowledge assistants.

RAG reduces but does not eliminate hallucination, meaning confident but wrong answers. Retrieval quality is usually the bottleneck: badly chunked or outdated documents produce bad answers.

Where it bites: access control (users must only retrieve what they are allowed to see), evaluation (a test set of real questions with known answers), and keeping the document set current.