Retrieval Augmented Generation (RAG) enhances large language models by retrieving relevant external documents at query time, grounding responses in factual, up-to-date information without retraining the model. RAG addresses core LLM limitations — hallucinations, outdated knowledge, and lack of domain-specific data — making it the most widely deployed pattern for production AI agent systems.

RAG operates in three stages:

1. Retrieval — A query is embedded into a vector $\mathbf{q} = E(\text{query})$ and used to search a knowledge base (vector database, keyword index, or hybrid) for the top-$k$ relevant document chunks by similarity $\text{sim}(\mathbf{q}, \mathbf{d}_i)$
2. Augmentation — Retrieved chunks are injected into the LLM prompt alongside the user query to provide grounding context
3. Generation — The LLM synthesizes a response $P(\text{answer} \mid \text{query}, d_1, d_2, \ldots, d_k)$ using both its training knowledge and the retrieved context

The simplest implementation: embed query, retrieve top-$k$ chunks by cosine similarity $\frac{\mathbf{q} \cdot \mathbf{d}}{||\mathbf{q}|| \cdot ||\mathbf{d}||}$, stuff into prompt, generate. Prone to retrieval noise, irrelevant chunks, and context overflow on complex queries.

Optimizes each stage of the pipeline:

• Pre-retrieval — Query rewriting (HyDE, ITER-RETGEN), query expansion, and decomposition for complex questions
• Retrieval — Hybrid search combining semantic vectors with BM25 keyword matching (which scores via $\text{BM25}(q, d) = \sum_{t \in q} \text{IDF}(t) \cdot \frac{f(t,d) \cdot (k_1 + 1)}{f(t,d) + k_1 \cdot (1 - b + b \cdot \frac{|d|}{\text{avgdl}})}$), plus fine-tuned embedding models
• Post-retrieval — Reranking retrieved results (Cohere Rerank, cross-encoders), context compression, and deduplication

Breaks the pipeline into interchangeable components for maximum flexibility:

• Iterative retrieval — Multiple retrieval rounds that refine results (RETRO, GAR-meets-RAG)
• Recursive retrieval — Multi-hop reasoning for questions requiring chain-of-thought across documents (IRCoT)
• Adaptive retrieval — The system decides when retrieval is needed versus when the LLM can answer directly (Self-RAG, FLARE)

Microsoft's GraphRAG builds a knowledge graph from documents, extracting entities and relationships, then uses graph traversal combined with vector search. This captures hierarchical context and entity connections that flat vector search misses, excelling on complex analytical queries.

from langchain_openai import OpenAIEmbeddings, ChatOpenAI
from langchain_community.vectorstores import Chroma
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain_cohere import CohereRerank
from langchain.retrievers import ContextualCompressionRetriever
 
# 1. Chunk documents with recursive splitting
splitter = RecursiveCharacterTextSplitter(
    chunk_size=512, chunk_overlap=64,
    separators=["\n\n", "\n", ". ", " "]
)
chunks = splitter.split_documents(documents)
 
# 2. Embed and store in vector database
embeddings = OpenAIEmbeddings(model="text-embedding-3-large")
vectorstore = Chroma.from_documents(chunks, embeddings)
 
# 3. Hybrid retrieval with reranking
base_retriever = vectorstore.as_retriever(search_kwargs={"k": 20})
reranker = CohereRerank(top_n=5)
retriever = ContextualCompressionRetriever(
    base_compressor=reranker,
    base_retriever=base_retriever
)
 
# 4. Generate with retrieved context
llm = ChatOpenAI(model="gpt-4")
docs = retriever.invoke("How does GraphRAG improve retrieval?")
context = "\n".join(doc.page_content for doc in docs)
response = llm.invoke(f"Context: {context}\n\nQuestion: How does GraphRAG improve retrieval?")

RAGAS (Retrieval Augmented Generation Assessment Suite) provides standard metrics for evaluating RAG pipelines:

• Faithfulness — Are generated claims supported by retrieved context? Measured as $\frac{|\text{supported claims}|}{|\text{total claims}|}$
• Answer relevance — Does the response address the actual question?
• Context precision — How much of the retrieved context is relevant? $\text{Precision@}k = \frac{|\text{relevant chunks in top-}k|}{k}$
• Context recall — Were all necessary documents retrieved? $\text{Recall} = \frac{|\text{relevant chunks retrieved}|}{|\text{total relevant chunks}|}$

• Prompting Guide - RAG Research
• Microsoft GraphRAG
• RAGAS Evaluation Framework
• Wikipedia - Retrieval Augmented Generation

• Embeddings — Embedding models that power RAG retrieval
• Knowledge Graphs — Graph-based retrieval with GraphRAG
• Agent Memory Frameworks — Memory systems that build on RAG patterns
• vector_databases — Storage infrastructure for RAG