Terminal-Bench 4.0
An agentic benchmark measuring whether an AI model can complete real command-line / terminal software tasks end-to-end inside sandboxed containers, scored by resolution rate. Version 4.0 (released 2026-08-28) is a 66-task set that calibrates per-task resources (time, CPU, memory), applies fixes to 19 tasks, removes 8 tasks from 3.0 (saturation, refusals, published solutions, platform issues), and sets a flat 8-hour agent timeout. A different task set from Terminal-Bench 2.0 / 2.1, so scores are not comparable across versions.
What is Terminal-Bench 4.0?
An agentic benchmark measuring whether an AI model can complete real command-line / terminal software tasks end-to-end inside sandboxed containers, scored by resolution rate. Version 4.0 (released 2026-08-28) is a 66-task set that calibrates per-task resources (time, CPU, memory), applies fixes to 19 tasks, removes 8 tasks from 3.0 (saturation, refusals, published solutions, platform issues), and sets a flat 8-hour agent timeout. A different task set from Terminal-Bench 2.0 / 2.1, so scores are not comparable across versions. evals.report tracks reported Terminal-Bench 4.0 scores with the model, source, status, date, and run caveats attached — official leaderboard scores, vendor-reported launches, and clearly labeled community runs.
Top reported Terminal-Bench 4.0 score: Codex + GPT-6 Astra — 58.2% (resolution rate).
| Model | Lab | Score↓ | Source model | Status | Date | |
|---|---|---|---|---|---|---|
| Codex + GPT-6 Astra | Agent systems | 58.2% | Codex + GPT-6 Astra | Official | — | Details |
| Claude Code + Claude Fable 5.1 | Agent systems | 57.9% | Claude Code + Fable 5.1 | Official | — | Details |
| Claude Code + Claude Opus 5 | Agent systems | 53.9% | Claude Code + Opus 5 | Official | — | Details |
| Claude Code + Claude Fable 5 | Agent systems | 44.5% | Claude Code + Fable 5 | Official | — | Details |
| Claude Code + GLM-5.3 | Agent systems | 41.8% | Claude Code + GLM-5.3 | Official | — | Details |
| Grok Build + Grok 4.7 | Agent systems | 37.6% | Grok Build + Grok 4.7 | Official | — | Details |
| Codex + GPT-5.6 Sol | Agent systems | 37.3% | Codex + GPT-5.6 Sol | Official | — | Details |
| Claude Code + Claude Opus 4.8 | Agent systems | 23.6% | Claude Code + Opus 4.8 | Official | — | Details |
| Codex + GPT-5.6 Terra | Agent systems | 21.5% | Codex + GPT-5.6 Terra | Official | — | Details |
| Grok Build + Grok 4.6 | Agent systems | 20.3% | Grok Build + Grok 4.6 | Official | — | Details |
| mini-SWE-agent + Gemini 3.8 Flash | Agent systems | 19.1% | mini-SWE-agent + Gemini 3.8 Flash | Official | — | Details |
| Codex + GPT-5.6 Luna | Agent systems | 17.3% | Codex + GPT-5.6 Luna | Official | — | Details |
| Grok Build + Grok 4.5 | Agent systems | 12.4% | Grok Build + Grok 4.5 | Official | — | Details |
| Claude Code + Claude Sonnet 5 | Agent systems | 12.4% | Claude Code + Sonnet 5 | Official | — | Details |
| mini-SWE-agent + Gemini 3.7 Flash | Agent systems | 11.2% | mini-SWE-agent + Gemini 3.7 Flash | Official | — | Details |
Each row reports the model’s resolution rate on Terminal-Bench 4.0. Click a row for the full run context.