Cognitive Companion is a research system designed to detect and monitor degradation in AI reasoning processes through specialized monitoring mechanisms. The system employs either large language model (LLM) judges or hidden-state probes positioned at specific model layers to identify performance decline in reasoning tasks, achieving high detection accuracy with minimal computational overhead.

Reasoning degradation in large language models represents a significant challenge in maintaining model reliability and performance consistency over extended deployments. As models encounter diverse queries and operating conditions, their reasoning quality may decline due to factors such as distribution shift, accumulated errors in reasoning chains, or task-specific performance drift. Cognitive Companion addresses this challenge by providing real-time monitoring capabilities that can detect when a model's reasoning processes begin to degrade, enabling proactive intervention or model recalibration ¹⁾

The system implements two complementary detection approaches for identifying reasoning degradation. The first approach utilizes LLM judges—separate language models trained to evaluate the quality and correctness of reasoning processes in the monitored system. These judges assess reasoning patterns, logical consistency, and solution validity. The second approach employs hidden-state probes that operate directly on model activations, specifically targeting layer-28 of the monitored model architecture. Hidden-state probes extract interpretable signals from intermediate model representations without requiring external evaluation ²⁾.

The probe-based approach achieves 0.840 AUROC (Area Under the Receiver Operating Characteristic Curve) in zero-overhead detection scenarios, meaning the system can identify reasoning degradation with high discriminative accuracy without adding computational cost to normal inference operations ³⁾.

Cognitive Companion demonstrates significant practical benefits in addressing reasoning repetition—a common degradation pattern where models begin to repeat reasoning steps or loop through similar inference paths. The system achieves 52-62% reduction in repetition when deployed as a mitigation mechanism ⁴⁾.

The computational overhead for maintaining these monitoring capabilities reaches approximately 11% of baseline inference cost, representing a reasonable trade-off between detection capability and system efficiency. This overhead encompasses the continuous probing of hidden states or parallel evaluation by LLM judges during inference ⁵⁾.

The layer-28 targeting in hidden-state probes suggests that meaningful degradation signals emerge at relatively deep layers of transformer-based models, where abstract reasoning representations have been substantially processed. This specificity allows the system to focus computational resources on layers most likely to contain diagnostic information about reasoning quality, rather than monitoring all layer activations ⁶⁾.

The dual-approach design provides flexibility in deployment scenarios. Organizations with access to capable separate LLM judges may prefer the external evaluation approach, while systems with computational constraints or latency requirements may benefit from the hidden-state probing method's direct activation analysis.

Cognitive Companion enables several practical applications in production AI systems. Real-time degradation detection allows for dynamic model switching, where systems can identify when reasoning quality declines and route requests to alternative models or trigger retraining procedures. The repetition reduction capabilities specifically address a common failure mode in chain-of-thought reasoning, where models become stuck in circular reasoning patterns.

The system may be particularly valuable in long-horizon reasoning tasks, where accumulated errors in reasoning steps can compound over extended inference sequences. By providing early warning of degradation onset, the system enables corrective action before failures propagate through complex reasoning chains.

• Cognitive Degradation Monitoring
• Reasoning Degradation Monitoring
• Chain of Thought Agents
• Devin: Autonomous AI Coding Agent
• Advanced Reasoning and Planning