Agentic Uncertainty

Uncertainty quantification (UQ) in LLM agents is a critical and underexplored challenge. Unlike single-turn question answering, agentic workflows involve sequential decisions where errors compound – each step's uncertainty inherits from and amplifies prior steps. This page covers the “Spiral of Hallucination” phenomenon and the UProp framework for principled uncertainty propagation.

In single-turn LLM interactions, uncertainty is localized to the current generation. In agentic systems, the agent operates in a partially observable environment where:

• The true environment state $s_t$ is latent
• The agent maintains a belief state $b_t(s_t) = P(s_t | h_t)$ based on history $h_t$
• Internal cognitive errors (hallucinations, logic gaps, memory failures) become external constraints for future actions
• Small early errors can cascade into catastrophic failures across the planning horizon

The Spiral of Hallucination (arXiv:2601.15703) identifies how early epistemic errors in LLM agents propagate irreversibly through the context window, creating a self-reinforcing cycle of degraded reasoning.

• Early errors (hallucinations, logic gaps, tool misinterpretation) propagate through the agent's context, biasing all subsequent reasoning steps
• Unlike single-step generation, agentic workflows suffer from state uncertainty – the agent's internal belief diverges from reality and this divergence compounds
• Self-reflection alone leads to “aimless corrections” that fail to halt the spiral
• Existing UQ methods act passively (diagnosing risk without intervention), which is insufficient for agentic settings

The paper models agent reliability failures as a Partially Observable Markov Decision Process (POMDP), distinguishing:

• Epistemic uncertainty: Reducible via better reasoning or knowledge – the agent's ignorance about the true state
• Aleatoric uncertainty: Irreducible randomness inherent in the environment

Inspired by dual-process theory from cognitive science:

• System 1 (Fast, forward propagation): Memory-aware uncertainty propagation that prevents error spread at each step – a proactive “immune system” for the agent's reasoning chain
• System 2 (Slow, inverse correction): Reflective calibration that detects deviations and applies targeted corrections before errors solidify into the context

This bridges passive sensing (traditional UQ) with active reasoning (intervention), improving long-horizon task performance, calibration, and self-awareness.

UProp (Duan et al., 2025) introduces a principled, information-theoretic framework for decomposing uncertainty in sequential agent decisions.

UProp decomposes LLM sequential decision uncertainty into two parts:

• Internal uncertainty: Intrinsic to the current decision – what existing single-turn UQ methods measure (e.g., token entropy, sampling variance)
• Extrinsic uncertainty: A Mutual Information (MI) quantity $I(d_t; d_{<t})$ describing how much uncertainty is inherited from preceding decisions

UProp efficiently estimates extrinsic uncertainty by converting direct MI estimation to Pointwise Mutual Information (PMI) estimation over multiple Trajectory-Dependent Decision Processes (TDPs):

$\hat{U}_{\text{extrinsic}}(d_t) = \text{PMI}(d_t; d_{<t}) = \log \frac{P(d_t | d_{<t})}{P(d_t)}$

This captures how much the current decision depends on (and inherits uncertainty from) the trajectory so far.

• Tested on AgentBench and HotpotQA with GPT-4.1 and DeepSeek-V3
• Significantly outperforms single-turn UQ baselines even when those baselines use thoughtful aggregation strategies
• Provides comprehensive analysis including sampling efficiency and intermediate uncertainty propagation

import numpy as np
 
class UncertaintyPropagator:
    """UProp-style uncertainty estimation for agentic decisions."""
 
    def __init__(self, llm, n_trajectories=10):
        self.llm = llm
        self.n_traj = n_trajectories
 
    def estimate_uncertainty(self, history, current_query):
        """Decompose uncertainty into internal + extrinsic components."""
        # Internal: uncertainty of current decision in isolation
        p_marginal = self.llm.sample_decisions(current_query, n=self.n_traj)
        internal = self._entropy(p_marginal)
 
        # Extrinsic: MI between current decision and trajectory history
        p_conditional = self.llm.sample_decisions(
            current_query, context=history, n=self.n_traj
        )
        pmi = np.log(p_conditional / (p_marginal + 1e-10) + 1e-10)
        extrinsic = np.mean(pmi)
 
        return {"internal": internal, "extrinsic": extrinsic,
                "total": internal + extrinsic}
 
    def should_intervene(self, uncertainty, threshold=0.7):
        """Dual-process: trigger System 2 correction if extrinsic
        uncertainty exceeds threshold."""
        return uncertainty["extrinsic"] > threshold
 
    @staticmethod
    def _entropy(probs):
        return -np.sum(probs * np.log(probs + 1e-10))

The degradation of agent confidence across sequential steps follows a characteristic pattern:

1. Step 1-2: Uncertainty is primarily internal; agent operates near its single-turn accuracy
2. Step 3-5: Extrinsic uncertainty begins to accumulate as earlier decisions constrain the solution space
3. Step 5+: The spiral effect – if early errors exist, belief state divergence accelerates non-linearly
4. Long horizon: Without intervention, total uncertainty can exceed the threshold for reliable operation

The key insight is that total agent uncertainty is not simply the sum of per-step uncertainties – the extrinsic (inherited) component introduces multiplicative compounding effects.

For a sequence of agent decisions $d_1, d_2, \ldots, d_T$:

$U_{\text{total}}(d_t) = \underbrace{H(d_t)}_\text{internal} + \underbrace{I(d_t; d_{<t})}_\text{extrinsic}$

where $H(d_t)$ is the entropy of the current decision and $I(d_t; d_{<t})$ is the mutual information with the trajectory prefix. The spiral of hallucination occurs when:

$I(d_t; d_{<t}) \gg H(d_t)$

i.e., the agent's current decision is dominated by inherited uncertainty from prior (potentially erroneous) steps rather than by its own reasoning capacity.

• Spiral of Hallucination: Uncertainty-Aware Agentic LLMs (arXiv:2601.15703)
• UProp: Uncertainty Propagation of LLMs in Multi-Step Agentic Decision-Making (arXiv:2506.17419)

• Spreading Activation Memory – memory architectures that may help mitigate context pollution
• Agent Red Teaming – exploiting uncertainty propagation as an attack vector
• LLM Theorem Proving – formal verification as a way to ground agent reasoning