This comparison examines the diagnostic accuracy of OpenAI's o1 large language model against experienced emergency department attending physicians in triage scenarios. A study involving 76 real emergency room cases demonstrated significant performance differences between AI-assisted diagnosis and traditional physician-led triage assessment.

The o1 model achieved 67% accuracy in emergency triage diagnoses across the 76-case evaluation set, substantially outperforming two experienced attending physicians who achieved 55% and 50% accuracy respectively ¹⁾. This 12-17 percentage point accuracy advantage represents a meaningful difference in clinical decision-making contexts where diagnostic precision directly impacts patient outcomes and resource allocation.

Notably, the performance evaluation methodology included blinded reviewer assessment, where reviewers could not distinguish between diagnoses generated by the AI system and those provided by human physicians. This controlled evaluation approach suggests the o1 model produces diagnostically plausible reasoning that aligns with professional clinical standards and terminology.

The performance gap between o1 and attending physicians was most pronounced during the early, information-sparse phase of triage assessment. Emergency medicine operates under severe time and information constraints, particularly in the initial minutes when clinical presentation is incomplete and diagnostic certainty remains low. In these high-stakes scenarios with minimal data—when patients first present with undifferentiated symptoms—the o1 model demonstrated relative advantages in diagnostic hypothesis generation and differential diagnosis formulation ²⁾.

This pattern suggests the o1 model may approach differential diagnosis generation differently than human physicians, possibly by leveraging extensive training data patterns that capture rare and atypical presentations. Traditional physician triage may be constrained by availability bias or anchoring on most common presentations, whereas the model may maintain broader consideration of lower-probability diagnoses when limited clinical information is available.

While the accuracy advantage shown by o1 is substantial, the triage context presents specific limitations requiring consideration. Emergency department decision-making involves not only diagnostic accuracy but also risk stratification, resource prioritization, and continuous monitoring protocols. The o1 evaluation focused on diagnostic accuracy in a retrospective case review format, which may not fully capture the dynamic nature of real-time emergency assessment where clinicians adjust diagnoses as new information emerges and patient status evolves.

Additionally, physician performance in the study may reflect scenarios where clinical context beyond presenting symptoms influences decision-making—intuitive pattern recognition developed through decades of practice may weigh factors that structured diagnostic algorithms do not explicitly model. The blinded review format, while methodologically sound, abstracts the diagnosis from the broader clinical context of ongoing assessment and management.

The o1 model's strength in information-sparse scenarios reflects its architectural approach to reasoning and chain-of-thought processing. The model generates extended deliberation about diagnostic possibilities before committing to a diagnosis, potentially allowing more comprehensive differential diagnosis generation than time-pressured human assessment in chaotic emergency environments ³⁾.

Attending physicians, by contrast, rely on pattern recognition honed through experience, rapid intuitive assessment refined by thousands of previous cases, and integration of non-linguistic clinical cues including patient presentation, tone of voice, and behavioral indicators. These capabilities remain difficult for large language models to access or appropriately weight in decision-making.

The study provides empirical evidence for potential clinical decision support applications, though diagnostic accuracy in retrospective case review differs from real-time clinical utility. Integration of o1-based diagnostic suggestions into emergency medicine workflows would require careful consideration of timing (when in the triage process recommendations are generated), transparency (how physicians understand the model's reasoning), and override protocols (how human judgment remains paramount in final decision-making).

• Harvard Medical AI Triage Study
• OpenAI o1 Model
• OpenAI vs Anthropic: Enterprise Deployment Strategies
• o3