evals.report
BenchmarksLabsCompareRun guidesIn the wild

Terminal-Bench 4.1

An agentic benchmark measuring whether an AI model can complete real command-line / terminal software tasks end-to-end inside sandboxed containers, scored by resolution rate. Version 4.1 (released 2026-11-09) keeps the 66 tasks of 4.0 and hardens their verifiers. Agent-written code now runs as an untrusted user in a separate RewardKit Worker, so it cannot read the tests or rewrite the reward file. Several overly strict verifiers were also fixed, for example music-harmony and freecad-platform-drawing. Terminal-Bench re-graded the saved 4.0 artifacts at no extra inference cost. 162 of the 11,550 trials changed score, and every agent's 4.1 score falls inside its 4.0 95% confidence interval. Trials without stored artifacts keep their 4.0 score.

Agentsresolution rateHigher is better

What this benchmark measures

An agentic benchmark measuring whether an AI model can complete real command-line / terminal software tasks end-to-end inside sandboxed containers, scored by resolution rate. Version 4.1 (released 2026-11-09) keeps the 66 tasks of 4.0 and hardens their verifiers. Agent-written code now runs as an untrusted user in a separate RewardKit Worker, so it cannot read the tests or rewrite the reward file. Several overly strict verifiers were also fixed, for example music-harmony and freecad-platform-drawing. Terminal-Bench re-graded the saved 4.0 artifacts at no extra inference cost. 162 of the 11,550 trials changed score, and every agent's 4.1 score falls inside its 4.0 95% confidence interval. Trials without stored artifacts keep their 4.0 score.

Rows on this page are sourced from public benchmark artifacts, leaderboard exports, or source-linked model reports. Each row keeps benchmark version, source model name, and available run details attached to the score.

The metric shown here is resolution rate. It should be interpreted within Terminal-Bench 4.1, not compared as part of a site-wide ranking.

What to be careful about

Stored separately from Terminal-Bench 4.0 because a different verifier set grades it, so a row here and a row there are not the same measurement even for the same run. Each scaffold+model appears once, at its best-scoring effort (a tie on solved trials goes to the higher effort).

No composite ranking
evals.report never combines benchmarks. resolution rate on Terminal-Bench 4.1 is its own number — don’t average it with other metrics.

Frequently asked

What is Terminal-Bench 4.1?

An agentic benchmark measuring whether an AI model can complete real command-line / terminal software tasks end-to-end inside sandboxed containers, scored by resolution rate. Version 4.1 (released 2026-11-09) keeps the 66 tasks of 4.0 and hardens their verifiers. Agent-written code now runs as an untrusted user in a separate RewardKit Worker, so it cannot read the tests or rewrite the reward file. Several overly strict verifiers were also fixed, for example music-harmony and freecad-platform-drawing. Terminal-Bench re-graded the saved 4.0 artifacts at no extra inference cost. 162 of the 11,550 trials changed score, and every agent's 4.1 score falls inside its 4.0 95% confidence interval. Trials without stored artifacts keep their 4.0 score. It is a agents benchmark measured by resolution rate.

What does resolution rate mean on Terminal-Bench 4.1?

Terminal-Bench 4.1 reports resolution rate (%); higher is better. Scores are shown only within Terminal-Bench 4.1 and are never averaged with other benchmarks.

What is the top reported Terminal-Bench 4.1 score?

Claude Code + Claude Opus 5.5 has the top reported score on Terminal-Bench 4.1: 65.8% (resolution rate).

Why do Terminal-Bench 4.1 scores differ across runs?

Harness, scaffold, reasoning effort, and prompt setup change results, so two runs of the same model can differ. evals.report keeps each score with its run context so the differences stay visible.

Does evals.report rank models across benchmarks?

No. Terminal-Bench 4.1 scores are shown within their own metric; evals.report never combines benchmarks into a composite ranking or a single "best model".