ZeroBench
An intentionally 'impossible' visual reasoning benchmark of 100 hand-crafted main questions (plus 334 subquestions) on which contemporary large multimodal models score near zero, designed to provide maximum headroom for measuring genuine multi-step visual understanding.
What this benchmark measures
An intentionally 'impossible' visual reasoning benchmark of 100 hand-crafted main questions (plus 334 subquestions) on which contemporary large multimodal models score near zero, designed to provide maximum headroom for measuring genuine multi-step visual understanding.
Rows on this page are sourced from public benchmark artifacts, leaderboard exports, or source-linked model reports. Each row keeps benchmark version, source model name, and available run details attached to the score.
The metric shown here is accuracy. It should be interpreted within ZeroBench, not compared as part of a site-wide ranking.
Frequently asked
What is ZeroBench?
An intentionally 'impossible' visual reasoning benchmark of 100 hand-crafted main questions (plus 334 subquestions) on which contemporary large multimodal models score near zero, designed to provide maximum headroom for measuring genuine multi-step visual understanding. It is a multimodal benchmark measured by accuracy.
What does accuracy mean on ZeroBench?
ZeroBench reports accuracy (%); higher is better. Scores are shown only within ZeroBench and are never averaged with other benchmarks.
What is the top reported ZeroBench score?
GPT-5.4 has the top reported score on ZeroBench: 23.0% (pass@5) (accuracy).
Why do ZeroBench scores differ across runs?
Harness, scaffold, reasoning effort, and prompt setup change results, so two runs of the same model can differ. evals.report keeps each score with its run context so the differences stay visible.
Does evals.report rank models across benchmarks?
No. ZeroBench scores are shown within their own metric; evals.report never combines benchmarks into a composite ranking or a single "best model".