This comparison examines the performance differential between Claude Opus 4.6 when optimized with the ForgeCode harness versus the Capy harness on the Terminal-Bench 2.0 evaluation benchmark. The evaluation demonstrates how post-training optimization techniques can produce measurable performance variations in large language models across different implementation frameworks.1)
Claude Opus 4.6 represents a significant iteration in Anthropic's model development pipeline. When evaluated on Terminal-Bench 2.0—a standardized benchmark for assessing terminal and command-line task performance—the model achieved materially different results depending on which post-training harness was employed during optimization. The ForgeCode harness produced an accuracy rate of 79.8%, while the Capy harness achieved 75.3%, yielding a performance differential of 4.5 percentage points 2).
The ForgeCode harness optimization approach yielded Claude Opus 4.6's superior performance on Terminal-Bench 2.0 with a measured accuracy of 79.8%. This harness likely incorporates specialized post-training techniques designed to enhance the model's capability in code-related reasoning, command execution planning, and terminal operation comprehension. Post-training optimization frameworks such as those employing reinforcement learning from human feedback (RLHF) or supervised fine-tuning (SFT) can substantially impact downstream task performance by aligning model behavior with specific evaluation criteria 3).
The 79.8% accuracy achieved through ForgeCode represents the higher end of the performance spectrum tested, suggesting that this harness's optimization strategy may be particularly well-aligned with Terminal-Bench 2.0's evaluation methodology and task distribution.
The Capy harness produced a measurable performance decrease, with Claude Opus 4.6 achieving 75.3% accuracy on the same Terminal-Bench 2.0 benchmark. This represents a 4.5 percentage point reduction relative to ForgeCode. While 75.3% demonstrates substantial capability in terminal operation tasks, the performance gap indicates that post-training optimization choices create meaningful divergence in model behavior across functionally similar frameworks. Such variations may arise from differences in training data selection, loss function design, or the specific behavioral objectives embedded within each harness's optimization process.
The performance differential between ForgeCode and Capy highlights a critical dimension of modern large language model development: post-training optimization specificity. Rather than representing fundamental differences in model architecture or base parameters, the 4.5 percentage point gap demonstrates that how models are optimized during post-training stages directly impacts their performance on standardized evaluations 4).
This phenomenon reflects broader trends in model development where multiple post-training approaches can be applied to the same base model, each producing distinct performance profiles on particular task categories. Organizations may implement different harnesses based on their specific operational requirements, whether optimizing for code generation accuracy, following procedural instructions, or balancing multiple capability dimensions.
The ForgeCode versus Capy comparison has significant implications for practitioners selecting between model implementations. When deploying Claude Opus 4.6 for terminal-related tasks, code execution, or command-line operations, the choice of post-training harness becomes materially relevant. The 79.8% versus 75.3% accuracy differential suggests that stakeholders prioritizing terminal operation accuracy should preferentially select the ForgeCode-optimized variant.
However, comprehensive model selection should consider factors beyond single-benchmark performance, including task-specific accuracy metrics, latency requirements, computational resource utilization, and whether specific harnesses have been optimized for distinct capability areas outside Terminal-Bench 2.0's scope.
Terminal-Bench 2.0 serves as a standardized evaluation framework for assessing large language model performance on terminal-related tasks, including command generation, shell operation comprehension, and system administration tasks. The benchmark's design enables comparative analysis of how different models and post-training approaches handle procedural command execution and system interaction reasoning. Performance on this benchmark indicates a model's suitability for use cases involving automated system administration, infrastructure-as-code applications, and terminal-based automation workflows.