Memory augmentation strategies are techniques that extend an AI agent's ability to retain and recall information beyond the constraints of a single context window. These approaches include vector-based retrieval systems, structured knowledge stores, episodic memory buffers, and hybrid architectures that combine multiple memory types. Effective memory augmentation enables agents to maintain coherence across long interactions, learn from past experiences, and build cumulative knowledge over time.

RAG remains the foundational memory augmentation strategy in 2025, though its core technology has matured and innovation now focuses on architectural hybridization rather than the retrieval mechanism itself.

Standard RAG encodes documents into embeddings $\mathbf{d}_i = E(\text{doc}_i)$, stores them in a vector database, and retrieves the most relevant chunks at query time by computing $\text{sim}(\mathbf{q}, \mathbf{d}_i) = \frac{\mathbf{q} \cdot \mathbf{d}_i}{||\mathbf{q}|| \cdot ||\mathbf{d}_i||}$ to inject into the LLM context. This grounds the model responses in external knowledge, reducing hallucinations and enabling access to information beyond the training data. Key infrastructure includes FAISS, HNSW-based vector databases (Pinecone, Weaviate, Milvus), and embedding models (OpenAI ada-002, Cohere embed-v3, BGE).

Hybrid RAG combines dense vector retrieval with sparse keyword search (BM25), metadata filtering, and structured queries. This addresses RAG weakness with exact-match queries (names, codes, dates) while retaining semantic flexibility. Weaviate and Elasticsearch both support hybrid search natively.

MIRIX (2025) demonstrates the frontier of RAG augmentation: an eight-agent, six-memory-type system that achieved 59.5% accuracy (+35% over traditional RAG) on multimodal tasks by integrating memory across screenshots, dialogue, and structured data while reducing storage by an order of magnitude.

Raw memories must be consolidated into useful, compact representations to prevent unbounded growth and maintain retrieval quality:

Episodic Summarization compresses detailed interaction histories into concise summaries while preserving key details. Rather than storing every conversation turn, the agent periodically generates summaries that capture the essential information, decisions, and outcomes.

Trust and Persistence Scoring tracks confidence levels in stored information. MARK uses domain-aligned, continually updating refined memory that actively suppresses hallucinations while promoting factuality across multi-agent systems.¹⁾ Information with low trust scores is deprioritized during retrieval.

Frequency-Based Recall Promotion amplifies frequently accessed information while deprioritizing noise. Memories that are repeatedly retrieved are promoted to higher tiers in hierarchical architectures, while rarely accessed items decay.

Salience Thresholds dynamically weight memories by relevance to the current task context. Rather than treating all stored information equally, the agent evaluates each memory relevance before including it in the context window.

Counterintuitively, intelligent forgetting often improves agent performance by reducing noise and focusing retrieval:

Time-Decay Functions gradually reduce the retrieval weight of older memories unless they are explicitly refreshed by access or reinforcement. A common model uses exponential decay: $w(t) = w_0 \cdot e^{-\lambda t}$, where $w_0$ is the initial weight, $\lambda$ is the decay rate, and $t$ is time since last access. This mirrors the natural decay of unused information in human memory.

LRU (Least Recently Used) Eviction removes the least recently accessed items when memory capacity limits are approached. Simple but effective for managing bounded memory stores.

Ebbinghaus Forgetting Curves. SAGE pioneered applying Ebbinghaus forgetting curves to agent memory.²⁾ The retention probability is modeled as:

$$R(t) = e^{-t/S}$$

where $R$ is retention, $t$ is time elapsed, and $S$ is the memory strength (increased by each review). Items are scheduled for review or deletion based on their predicted retention probability.

Attention-Weighted Persistence uses the attention scores from retrieval events to determine which memories are most valuable. Items that consistently receive high attention when retrieved are protected from eviction; items that are retrieved but receive low attention are candidates for pruning.

The most effective 2025 systems combine multiple memory paradigms:

CDMem (Gao et al., 2025) implements three-stage hierarchical encoding through graph-structured, context-dependent indexing with expert, short-term, and long-term memory layers. It achieved 85.8% success on ALFWorld and 56.0% on ScienceWorld by enabling multilevel knowledge recall tailored to current contexts.

LM2 adds gated memory modules to each transformer decoder layer, outperforming standard transformers on multi-hop reasoning over 128K-token contexts.³⁾ The gated mechanism learns when to write to and read from external memory during the forward pass, controlled by a gate $g = \sigma(\mathbf{W}_g \mathbf{h} + \mathbf{b}_g)$ that modulates the memory output.

UserCentrix employs Value of Information (VoI) gating and hierarchical control to balance efficiency and personalization, achieving 2x accuracy over no-memory baselines while reducing resource consumption. The VoI mechanism evaluates whether retrieving additional memory is worth the latency cost for each query, computing $\text{VoI} = \mathbb{E}[\text{Accuracy gain}] - \text{Cost}_{\text{retrieval}}$.

import numpy as np
from [[openai|openai]] import [[openai|OpenAI]]
 
client = [[openai|OpenAI]]()
 
DOCS = [
    "RAG retrieves relevant documents and injects them into the LLM context window.",
    "Vector databases store [[embeddings|embeddings]] for semantic similarity search using ANN algorithms.",
    "Re-ranking reorders initial retrieval results using a cross-encoder for higher precision.",
    "BM25 is a sparse retrieval method based on term frequency and inverse document frequency.",
    "[[hybrid_search|Hybrid search]] combines dense vector retrieval with sparse keyword matching.",
    "Fine-tuning adapts a pre-trained model to a specific domain using labeled data.",
    "[[prompt_engineering|Prompt engineering]] designs input templates to elicit desired model behavior.",
]
 
 
def get_embeddings(texts: list[str]) -> np.ndarray:
    response = client.[[embeddings|embeddings]].create(model="text-embedding-3-small", input=texts)
    return np.array([e.embedding for e in response.data])
 
 
def cosine_similarity(a: np.ndarray, b: np.ndarray) -> np.ndarray:
    norms_a = np.linalg.norm(a, axis=1, keepdims=True)
    norms_b = np.linalg.norm(b, axis=1, keepdims=True)
    return (a @ b.T) / (norms_a * norms_b.T)
 
 
def retrieve_and_rerank(query: str, docs: list[str], top_k: int = 3) -> list[str]:
    # Stage 1: Dense retrieval
    doc_embeds = get_embeddings(docs)
    query_embed = get_embeddings([query])
    scores = cosine_similarity(query_embed, doc_embeds)[0]
    candidates = sorted(range(len(docs)), key=lambda i: scores[i], reverse=True)[:top_k + 2]
 
    # Stage 2: LLM re-ranking
    candidate_texts = [docs[i] for i in candidates]
    rerank_prompt = (
        f"Query: {query}\n\nRank these documents by relevance (most relevant first). "
        f"Return only the numbers, comma-separated:\n"
        + "\n".join(f"{i+1}. {doc}" for i, doc in enumerate(candidate_texts))
    )
    response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[{"role": "user", "content": rerank_prompt}],
        temperature=0.0,
    )
    ranking = [int(x.strip()) - 1 for x in response.choices[0].message.content.split(",") if x.strip().isdigit()]
    reranked = [candidate_texts[i] for i in ranking if i < len(candidate_texts)]
    return reranked[:top_k]
 
 
results = retrieve_and_rerank("How does hybrid retrieval improve search?", DOCS)
for i, doc in enumerate(results, 1):
    print(f"{i}. {doc}")

A major 2025 advance is memory systems that actively evaluate and update their own contents:

Reflection-Based Updating. MARK and SAGE use iterative reflection cycles where agents evaluate stored information accuracy and relevance, then dynamically adjust persistence scores. This prevents stale or contradictory information from accumulating over time.

Transformer-Squared enables real-time task adaptation by encoding procedural expertise into the parameter space using SVD decomposition of feedforward layers ($\mathbf{W} = \mathbf{U} \mathbf{\Sigma} \mathbf{V}^\top$). During inference, specialized expert vectors are dynamically blended via MLP mixing networks, adapting the model implicit memory without traditional fine-tuning.⁴⁾

Titans demonstrates that optimal context memorization enables complex reasoning by learning which historical information remains relevant for current inferences, using a neural long-term memory module that helps attention attend to the current context while utilizing long past information.⁵⁾

Beyond external memory stores, researchers have augmented the transformer architecture itself with memory capabilities:

The Memorizing Transformer uses $k$NN-retrievable external memory to dynamically integrate distant context, scaling to 262K tokens while outperforming baselines on long-range reasoning tasks including theorem proving and code generation.⁶⁾ The $k$NN mechanism mirrors hippocampal episodic retrieval in neuroscience.

Gated Memory Integration adds learnable gates to attention layers that control information flow between the model internal representations and external memory stores. The output at each layer is computed as $\mathbf{o} = g \cdot \mathbf{m} + (1 - g) \cdot \mathbf{h}$, where $g$ is the gate value, $\mathbf{m}$ is the memory output, and $\mathbf{h}$ is the standard attention output. This allows the model to seamlessly blend parametric knowledge (weights) with retrieved knowledge (external memory) at each layer.

Memory-augmented transformers can be categorized along three dimensions: functional objectives (context extension, reasoning, knowledge integration, adaptation), memory representations (parameter-encoded, state-based, explicit, hybrid), and integration mechanisms (attention fusion, gated control, associative retrieval).

• How to Add Memory to an Agent
• Memory Retention
• Agent Memory Frameworks
• Long-Term Memory
• Hermes Four-Layer Memory vs Context-Window + RAG