Terminal-Bench 4.1
An agentic benchmark measuring whether an AI model can complete real command-line / terminal software tasks end-to-end inside sandboxed containers, scored by resolution rate. Version 4.1 (released 2026-11-09) keeps the 66 tasks of 4.0 and hardens their verifiers. Agent-written code now runs as an untrusted user in a separate RewardKit Worker, so it cannot read the tests or rewrite the reward file. Several overly strict verifiers were also fixed, for example music-harmony and freecad-platform-drawing. Terminal-Bench re-graded the saved 4.0 artifacts at no extra inference cost. 162 of the 11,550 trials changed score, and every agent's 4.1 score falls inside its 4.0 95% confidence interval. Trials without stored artifacts keep their 4.0 score.
What is Terminal-Bench 4.1?
An agentic benchmark measuring whether an AI model can complete real command-line / terminal software tasks end-to-end inside sandboxed containers, scored by resolution rate. Version 4.1 (released 2026-11-09) keeps the 66 tasks of 4.0 and hardens their verifiers. Agent-written code now runs as an untrusted user in a separate RewardKit Worker, so it cannot read the tests or rewrite the reward file. Several overly strict verifiers were also fixed, for example music-harmony and freecad-platform-drawing. Terminal-Bench re-graded the saved 4.0 artifacts at no extra inference cost. 162 of the 11,550 trials changed score, and every agent's 4.1 score falls inside its 4.0 95% confidence interval. Trials without stored artifacts keep their 4.0 score. evals.report tracks reported Terminal-Bench 4.1 scores with the model, source, status, date, and run caveats attached — official leaderboard scores, vendor-reported launches, and clearly labeled community runs.
Top reported Terminal-Bench 4.1 score: Claude Code + Claude Opus 5.5 — 65.8% (resolution rate).
| Model | Lab | Score↓ | Source model | Status | Date | |
|---|---|---|---|---|---|---|
| Claude Code + Claude Opus 5.5 | Agent systems | 65.8% | Claude Code + Opus 5.5 | Official | — | Details |
| Claude Code + Claude Fable 5.2 | Agent systems | 65.8% | Claude Code + Fable 5.2 | Official | — | Details |
| Antigravity CLI + Gemini 4 Carbon | Agent systems | 64.5% | Antigravity CLI + Gemini 4 Carbon | Official | — | Details |
| Claude Code + Claude Sonnet 5.5 | Agent systems | 63.9% | Claude Code + Sonnet 5.5 | Official | — | Details |
| Codex + GPT-6.1 Sol | Agent systems | 60.9% | Codex + GPT-6.1 Sol | Official | — | Details |
| Codex + GPT-6 Astra | Agent systems | 60.3% | Codex + GPT-6 Astra | Official | — | Details |
| Claude Code + Kimi K3.1 Pro | Agent systems | 59.7% | Claude Code + Kimi K3.1 Pro | Official | — | Details |
| Claude Code + Claude Fable 5.1 | Agent systems | 57.6% | Claude Code + Fable 5.1 | Official | — | Details |
| Antigravity CLI + Gemini 4 Hydrogen | Agent systems | 56.4% | Antigravity CLI + Gemini 4 Hydrogen | Official | — | Details |
| Antigravity CLI + Gemini 4 Argon | Agent systems | 55.5% | Antigravity CLI + Gemini 4 Argon | Official | — | Details |
| Claude Code + Claude Opus 5 | Agent systems | 53.3% | Claude Code + Opus 5 | Official | — | Details |
| Codex + GPT-6.1 Terra | Agent systems | 53.3% | Codex + GPT-6.1 Terra | Official | — | Details |
| Codex + GPT-6 Sol | Agent systems | 50.3% | Codex + GPT-6 Sol | Official | — | Details |
| Muse Code + Muse Wonder | Agent systems | 47.0% | Muse Code + Muse Wonder | Official | — | Details |
| Claude Code + Claude Fable 5 | Agent systems | 44.8% | Claude Code + Fable 5 | Official | — | Details |
| Claude Code + GLM-5.3 | Agent systems | 41.2% | Claude Code + GLM-5.3 | Official | — | Details |
| Claude Code + Claude Haiku 5.5 | Agent systems | 40.0% | Claude Code + Haiku 5.5 | Official | — | Details |
| Grok Build + Grok 4.7 | Agent systems | 38.8% | Grok Build + Grok 4.7 | Official | — | Details |
| Codex + GPT-5.6 Sol | Agent systems | 38.5% | Codex + GPT-5.6 Sol | Official | — | Details |
| Claude Code + DeepSeek V4.1 Flash 1017 | Agent systems | 38.5% | Claude Code + DeepSeek V4.1 Flash 1017 | Official | — | Details |
| Claude Code + Hy4 | Agent systems | 37.9% | Claude Code + Hy4 | Official | — | Details |
| Claude Code + GLM-5.3-Flash | Agent systems | 37.0% | Claude Code + GLM-5.3-Flash | Official | — | Details |
| Muse Code + Muse Ember | Agent systems | 30.6% | Muse Code + Muse Ember | Official | — | Details |
| Claude Code + Mistral Large 4 Preview | Agent systems | 27.6% | Claude Code + Mistral Large 4 | Official | — | Details |
| Claude Code + Qwen3.8 Max (0902) | Agent systems | 27.0% | Claude Code + Qwen3.8-Max-0902 | Official | — | Details |
| mini-SWE-agent + MiMo-V2.6-Pro | Agent systems | 27.0% | mini-swe-agent + MiMo V2.6 Pro | Official | — | Details |
| Claude Code + Claude Opus 4.8 | Agent systems | 23.6% | Claude Code + Opus 4.8 | Official | — | Details |
| Claude Code + Kimi K3 | Agent systems | 23.0% | Claude Code + Kimi K3 | Official | — | Details |
| Codex + GPT-5.6 Terra | Agent systems | 21.5% | Codex + GPT-5.6 Terra | Official | — | Details |
| Grok Build + Grok 4.6 | Agent systems | 19.7% | Grok Build + Grok 4.6 | Official | — | Details |
| mini-SWE-agent + Gemini 3.8 Flash | Agent systems | 18.2% | mini-SWE-agent + Gemini 3.8 Flash | Official | — | Details |
| Codex + GPT-5.6 Luna | Agent systems | 17.6% | Codex + GPT-5.6 Luna | Official | — | Details |
| mini-SWE-agent + MiMo-V2.6-Flash | Agent systems | 16.1% | mini-swe-agent + MiMo V2.6 Flash | Official | — | Details |
| Codex + GPT-6 Luna | Agent systems | 15.2% | Codex + GPT-6 Luna | Official | — | Details |
| Muse Code + Muse Spark 1.3 | Agent systems | 14.8% | Muse Code + Muse Spark 1.3 | Official | — | Details |
| mini-SWE-agent + Inkling 2 Small | Agent systems | 13.0% | mini-swe-agent + Inkling Small 2 | Official | — | Details |
| mini-SWE-agent + Gemini 3.7 Flash | Agent systems | 12.4% | mini-SWE-agent + Gemini 3.7 Flash | Official | — | Details |
| Grok Build + Grok 4.5 | Agent systems | 12.4% | Grok Build + Grok 4.5 | Official | — | Details |
| Claude Code + Claude Sonnet 5 | Agent systems | 12.1% | Claude Code + Sonnet 5 | Official | — | Details |
| mini-SWE-agent + MiniMax M3 | Agent systems | 7.6% | mini-swe-agent + MiniMax M3 | Official | — | Details |
| Claude Code + DeepSeek V4 Pro 0813 | Agent systems | 6.7% | Claude Code + DeepSeek V4 Pro 0813 | Official | — | Details |
| mini-SWE-agent + Beam 1.0 | Agent systems | 6.4% | mini-swe-agent + Beam 1.0 | Official | — | Details |
| Codex + GPT-5.4 mini | Agent systems | 1.8% | Codex + GPT-5.4 Mini | Official | — | Details |
| mini-SWE-agent + Inkling Small | Agent systems | 0.9% | mini-swe-agent + Inkling Small | Official | — | Details |
| mini-SWE-agent + Nemotron 3.5 Super VL | Agent systems | 0.3% | mini-swe-agent + Nemotron 3.5 Super VL | Official | — | Details |
| mini-SWE-agent + Inkling | Agent systems | 0.0% | mini-swe-agent + Inkling | Official | — | Details |
Each row reports the model’s resolution rate on Terminal-Bench 4.1. Click a row for the full run context.