Agent Planning: How AI Agents Plan and Reason

Agent planning is a critical capability that enables AI agents to devise efficient and effective solutions to complex, multi-step problems. AI agent planning encompasses the strategies and techniques that allow LLM-based agents to break down goals, sequence actions, and adapt on the fly. Rather than generating responses in a single pass, planning-capable agents decompose goals into sub-tasks, reason about action sequences, and adapt their strategies based on intermediate feedback.

For information on how agents store and retrieve context across interactions, see Memory Management in LLM Agents.

Introduced by Wei et al., 2022¹⁾, CoT prompting elicits step-by-step reasoning.²⁾ Variants include Zero-Shot CoT (“Let's think step by step”), Self-Consistency (majority voting over multiple reasoning paths), and Chain-of-Associated-Thoughts (CoAT, 2025) which integrates Monte Carlo Tree Search for exploring reasoning branches. See Advanced Reasoning and Planning for detailed coverage.

ReAct (Yao et al., 2022³⁾ combines reasoning and acting in an interleaved loop: the agent generates a thought (reasoning trace), takes an action (tool call), and observes the result.⁴⁾ This tight feedback loop enables dynamic replanning based on real-world outcomes. ReAct has become a standard pattern in frameworks like LangChain and LlamaIndex.

ToT (Yao et al., 2023⁵⁾ explores multiple reasoning paths simultaneously using tree search (BFS/DFS).⁶⁾ Each intermediate thought is evaluated for promise, allowing the agent to backtrack from unproductive branches. Effective for tasks requiring exploration such as puzzle-solving and creative writing.

GoT (Besta et al., 2024⁷⁾ generalizes planning to arbitrary directed graphs, enabling aggregation of partial solutions, refinement loops, and non-linear information flow.⁸⁾ A unified taxonomy by Besta et al. (2025) compares chains, trees, and graphs across cost-accuracy tradeoffs.

Modern frontier models function as end-to-end planners:

• OpenAI o3/o4-mini: Use extended chain-of-thought with reinforcement learning for inference-time compute scaling, enabling variable-depth planning
• Gemini 2.5 Pro: Combines multimodal reasoning with structured tool chains for complex workflows
• DeepSeek-R1: Open-weight model using RL-trained reasoning for planning tasks, competitive with proprietary models on standard PDDL domains (Liu et al., 2025)

A 2025 evaluation tested DeepSeek R1, Gemini 2.5 Pro, and GPT-5 against the LAMA planner on International Planning Competition domains. GPT-5 was competitive on standard tasks, but all LLMs degraded significantly on obfuscated domains requiring pure logical reasoning.

Combining LLMs with classical planners addresses reliability gaps. See LLM+P for the full treatment. Key approaches:

• LLM+P (Liu et al., 2023⁹⁾: LLM translates natural language to PDDL; classical planner (Fast Downward) solves it¹⁰⁾
• LLM as Planning Formalizer (Tantakoun, Muise, Zhu, 2025): LLMs construct and iteratively refine PDDL models for off-the-shelf planners
• MIT Optimization Integration (2025): Teaching LLMs optimization algorithms for multi-step planning challenges

World models simulate environment dynamics, allowing agents to “imagine” action consequences before executing them:

• Voyager (Wang et al., 2023¹¹⁾: LLM-driven agent in Minecraft that builds a persistent skill library through world-model-guided exploration¹²⁾
• DreamerV3 (Hafner et al., 2023¹³⁾: Learns world models from pixels for model-based RL planning
• 2025 work integrates world models with inference-time scaling (extended CoT) for embodied planning in robotics and logistics

LLM-based planning has extended to physical agents:

• SayCan (Ahn et al., 2022¹⁴⁾: Combines LLM semantic knowledge with learned affordance functions that ground plans in physical capabilities¹⁵⁾
• PaLM-E (Driess et al., 2023¹⁶⁾: Embodies a 562B parameter language model with visual and sensor inputs for multimodal robotic planning¹⁷⁾
• Inner Monologue (Huang et al., 2022¹⁸⁾: Uses LLM self-talk to reason through embodied tasks step by step, incorporating environmental feedback¹⁹⁾
• Code as Policies (Liang et al., 2022²⁰⁾: LLMs generate executable robot policy code directly from natural language instructions²¹⁾

from [[openai|openai]] import [[openai|OpenAI]]
 
client = [[openai|OpenAI]]()
 
DECOMPOSITION_PROMPT = """Break the following task into 3-7 concrete subtasks.
Return as a numbered list. Each subtask should be independently actionable.
 
Task: {task}"""
 
 
def decompose_task(task: str) -> list[str]:
    response = client.chat.completions.create(
        model="gpt-4o",
        messages=[{"role": "user", "content": DECOMPOSITION_PROMPT.format(task=task)}],
        temperature=0.2,
    )
    lines = response.choices[0].message.content.strip().split("\n")
    subtasks = []
    for line in lines:
        cleaned = line.strip().lstrip("0123456789.)- ").strip()
        if cleaned:
            subtasks.append(cleaned)
    return subtasks
 
 
def plan_and_execute(goal: str) -> dict:
    subtasks = decompose_task(goal)
    results = {}
    for i, subtask in enumerate(subtasks, 1):
        print(f"Step {i}: {subtask}")
        response = client.chat.completions.create(
            model="gpt-4o",
            messages=[
                {"role": "system", "content": "Complete the following subtask concisely."},
                {"role": "user", "content": subtask},
            ],
        )
        results[subtask] = response.choices[0].message.content
    return results
 
 
results = plan_and_execute("Build a REST API for a todo app with authentication")
for step, output in results.items():
    print(f"\n--- {step} ---\n{output[:200]}")

Static plans often fail in complex environments. Modern agents implement:

• Iterative Refinement: Executing a plan, observing outcomes, and revising subsequent steps
• Self-Refine (Madaan et al., 2023²²⁾: Using LLM self-feedback to improve plans before execution²³⁾
• Least-to-Most Prompting (Zhou et al., 2022²⁴⁾: Progressive decomposition from simple to complex sub-problems²⁵⁾
• RLVR (Reinforcement Learning with Verifiable Rewards): Post-training technique enabling verifiable planning in math and code domains, predicted to expand to broader planning in 2026

• PlanBench (Valmeekam et al., 2023²⁶⁾: Systematic evaluation of LLM planning on classical domains; 2025 updates show frontier models closing the gap to symbolic planners²⁷⁾
• TravelPlanner (Xie et al., 2024²⁸⁾: Complex real-world planning requiring constraint satisfaction across flights, hotels, and budgets²⁹⁾
• International Planning Competition (IPC): Standard PDDL domains used to compare LLMs against classical planners; obfuscated variants test pure reasoning
• WebArena (Zhou et al., 2023³⁰⁾: Web navigation tasks requiring multi-step planning³¹⁾
• SWE-Bench (Jimenez et al., 2024³²⁾: Software engineering tasks requiring planning across codebases³³⁾

Complex tasks increasingly use coordinated multi-agent planning:

• MAPF (Multi-Agent Path Finding): AAAI 2025 work on coordinating multiple agents in shared environments
• Hierarchical Multi-Agent Workflows (Liu et al., 2025, ICLR Workshop): Structured coordination for complex tasks
• Meta-Prompt Optimization (Kong et al., 2025, ICLR Workshop): Optimizing prompts for sequential multi-agent decisions

• Advanced Reasoning and Planning
• Multi-Agent Architecture (Planner-Generator-Evaluator)
• Agentic Engineering
• How to Create an Agent
• AI Agents

• Wei, J. et al. “Chain-of-Thought Prompting Elicits Reasoning in Large Language Models.” arXiv:2201.11903, 2022.
• Yao, S. et al. “ReAct: Synergizing Reasoning and Acting in Language Models.” arXiv:2210.03629, 2022.
• Yao, S. et al. “Tree of Thoughts: Deliberate Problem Solving with Large Language Models.” arXiv:2305.10601, 2023.
• Besta, M. et al. “Graph of Thoughts: Solving Elaborate Problems with Large Language Models.” arXiv:2308.11114, 2024.
• Liu, B. et al. “LLM+P: Empowering Large Language Models with Optimal Planning Proficiency.” arXiv:2304.11477, 2023.
• Wang, G. et al. “Voyager: An Open-Ended Embodied Agent with Large Language Models.” arXiv:2305.16291, 2023.
• Hafner, D. et al. “Mastering Diverse Domains through World Models.” arXiv:2301.04104, 2023.
• Ahn, M. et al. “Do As I Can, Not As I Say: Grounding Language in Robotic Affordances.” arXiv:2207.05916, 2022.
• Driess, D. et al. “PaLM-E: An Embodied Multimodal Language Model.” arXiv:2303.03378, 2023.
• Huang, W. et al. “Inner Monologue: Embodied Reasoning through Planning with Language Models.” arXiv:2207.05608, 2022.
• Liang, J. et al. “Code as Policies: Language Model Programs for Embodied Control.” arXiv:2209.07753, 2022.
• Madaan, A. et al. “Self-Refine: Iterative Refinement with Self-Feedback.” arXiv:2303.17651, 2023.
• Zhou, D. et al. “Least-to-Most Prompting Enables Complex Reasoning in Large Language Models.” arXiv:2210.11443, 2022.
• Valmeekam, K. et al. “PlanBench: An Extensible Benchmark for Evaluating Large Language Models on Planning and Reasoning about Change.” arXiv:2305.10918, 2023.
• Xie, J. et al. “TravelPlanner: A Benchmark for Real-World Planning with Language Agents.” arXiv:2403.12687, 2024.
• Zhou, S. et al. “WebArena: A Realistic Web Environment for Building Autonomous Agents.” arXiv:2307.13854, 2023.
• Jimenez, C. E. et al. “SWE-bench: Can Language Models Resolve Real-World GitHub Issues?” arXiv:2310.06770, 2024.