Continual learning agents are AI systems that improve from ongoing experience without losing previously acquired capabilities. Unlike traditional models trained once on static datasets, these agents accumulate skills, adapt to new tasks, and refine their behavior over extended deployment lifetimes while resisting catastrophic forgetting.

The central challenge of continual learning is the stability-plasticity dilemma: an agent must be plastic enough to learn new skills while stable enough to retain old ones. When neural networks are naively fine-tuned on new data, they catastrophically forget prior knowledge — weights optimized for old tasks are overwritten by gradients from new ones.

For LLM-based agents operating in dynamic environments, continual learning becomes critical. An agent that solves customer support tickets, writes code, or conducts research must accumulate domain knowledge over time rather than starting fresh with each interaction. The 2024-2025 research landscape addresses this through both parametric approaches (modifying model weights) and non-parametric approaches (managing external memory and context).

Catastrophic forgetting occurs when training on new tasks degrades performance on previously learned tasks. In the context of agents, this manifests as:

• Losing the ability to use tools the agent previously mastered
• Forgetting domain-specific procedures after learning new domains
• Degrading general reasoning quality after specialized fine-tuning
• Overwriting useful behavioral patterns with new but conflicting ones

Recent work distinguishes between unwanted forgetting (loss of useful knowledge) and adaptive unlearning (intentional deprecation of outdated or incorrect knowledge).

A-Mem introduces an agentic memory system for LLM agents that dynamically organizes memories using principles from the Zettelkasten method. When new memories are added, the system generates comprehensive notes with structured attributes including contextual descriptions, keywords, and tags. It then analyzes historical memories to identify relevant connections, establishing links where meaningful similarities exist.

The key innovation is memory evolution: as new memories are integrated, they trigger updates to contextual representations and attributes of existing memories, allowing the knowledge network to continuously refine its understanding. Experiments across six foundation models show superior performance over existing baselines.

# A-Mem style agentic memory system
class AgenticMemory:
    def __init__(self, llm, vector_store):
        self.llm = llm
        self.store = vector_store  # e.g., ChromaDB
        self.links = {}            # Memory connection graph
 
    def add_memory(self, experience):
        # Generate structured note with attributes
        note = self.llm.create_note(
            content=experience,
            attributes=["context", "keywords", "tags"]
        )
 
        # Find and link related historical memories
        related = self.store.search(note.embedding, top_k=10)
        for memory in related:
            if self.llm.assess_relevance(note, memory) > 0.7:
                self.links.setdefault(note.id, []).append(memory.id)
 
                # Memory evolution: update existing memory context
                memory.context = self.llm.refine_context(
                    memory, new_info=note
                )
                self.store.update(memory)
 
        self.store.add(note)
        return note

Letta proposes that LLM agents can achieve continual learning by updating context (prompts, history, memories) rather than model weights. This non-parametric approach enables perpetual learning across model generations without fine-tuning. Agents self-manage their memory through post-training context awareness, with optional distillation into parametric memory for efficiency. This sidesteps catastrophic forgetting entirely since model weights remain unchanged.

The Follow-The-Leader Online Agent uses shallow online world models with model predictive control for continual reinforcement learning. It achieves immunity to forgetting with provable regret bounds and outperforms deep learning models on the Continual Bench benchmark through incremental updates.

Experience replay stores past experiences and replays them during new learning to maintain old knowledge. Key variants include:

• GRASP — a rehearsal policy optimized for efficient online continual learning
• t-DGR — trajectory-based deep generative replay that simulates realistic past experiences for decision-making tasks
• Selective replay — prioritizing replay of experiences most at risk of being forgotten

Rather than fine-tuning all parameters, progressive approaches selectively update or add parameters:

• LoRA-based adaptation — low-rank updates that preserve base model knowledge
• Modular skill networks — separate parameter blocks for different capabilities
• Regularization methods — penalizing changes to parameters important for prior tasks

Agents build a growing library of reusable skills from experience:

• Sub-goal distillation — decomposing successful trajectories into transferable skills
• Skill indexing — tagging and retrieving skills based on task similarity
• Compositional generalization — combining existing skills to handle novel tasks

Two dominant paradigms have emerged for continual learning in agents:

Parametric (weight-based): Modify the model itself through careful fine-tuning with replay, regularization, or modular expansion. Offers deep integration but risks forgetting.

Non-parametric (context-based): Maintain external memory systems that provide relevant past experience through retrieval. Avoids forgetting but is limited by context window size and retrieval quality.

Hybrid approaches combine both: the agent uses non-parametric memory for immediate adaptation while periodically distilling frequently-used knowledge into parametric form.

• Continual Bench — evaluates agents across sequences of changing tasks
• CoLLAs benchmarks — standardized evaluations from the Conference on Lifelong Learning Agents
• CIRL tasks — continual inverse reinforcement learning evaluations
• Domain-incremental settings — measuring retention across sequential domain shifts

• Scalability — memory systems grow unbounded without effective forgetting mechanisms
• Evaluation — measuring knowledge retention across long deployment lifetimes is expensive
• Transfer vs. interference — new knowledge can help (forward transfer) or hurt (backward interference) prior skills
• Deployment constraints — production LLM agents often cannot be retrained, limiting parametric approaches
• Meta-cognition — agents need to know what they know and what they have forgotten

• Adaptation of Agentic AI (arXiv:2512.16301)
• A-Mem: Agentic Memory for LLM Agents (arXiv:2502.12110)
• A-Mem GitHub Repository
• Letta: Continual Learning for LLM Agents
• CoLLAs 2024 Proceedings (PMLR Vol. 274)
• Conference on Lifelong Learning Agents

• Long-Horizon Agents
• Agent Trajectory Optimization
• AI Scientist Agents
• Durable Execution for Agents