RAG means generating after retrieving information
Retrieval-augmented generation, or RAG, connects a language model to an external source. Before answering, the system searches for passages related to the question. It then adds those passages to the prompt so the model can write from them.
Google Cloud defines RAG as a combination of traditional search, databases and generative models. It can supply private, specialised or recent information that was absent from the original training data.
The full process in four steps
Documents are prepared, divided into chunks and stored in an index. Many systems turn each chunk into an embedding, a numerical representation of meaning. When a question arrives, retrieval finds close matches and may rerank them for relevance.
The model receives the question with that context and generates an answer. If the application preserves provenance, it can display citations too. RAG does not change the model's weights and is not retraining. It changes only the information available for that request.
What RAG solves and what it does not
RAG suits assistants built around manuals, regulations, catalogues or internal files. Updating an index is usually faster than retraining. It can also reduce how much text enters each request and make it easier to connect an answer to its original document.
It does not guarantee truth. Irrelevant, outdated or incorrectly permissioned retrieval can produce a confident mistake. Quality depends on chunking, indexing, queries, reranking and instructions. Our context-window guide explains why sending an entire library to a model is rarely sensible.
RAG, web search and fine-tuning are different
RAG retrieves information for a request. Web search is one possible source, but a private folder or database can serve the same role. Fine-tuning changes model behaviour through examples and better suits tone, format or a repeatable task than facts that change frequently.
A product may combine all three. Sensitive documents also require access controls before retrieval. Local and cloud AI determine where the system runs, while our guide to how ChatGPT works clarifies what the model does with supplied context.
The conversation starts here
Sign in with a supporter account to comment. Sign in
Nobody has commented yet. Want to go first?