SimpleQA Verified
A factual short-answer QA benchmark measuring parametric knowledge and hallucination resistance (Epoch AI's SimpleQA Verified).
What this benchmark measures
A factual short-answer QA benchmark measuring parametric knowledge and hallucination resistance (Epoch AI's SimpleQA Verified).
Rows on this page are sourced from public benchmark artifacts, leaderboard exports, or source-linked model reports. Each row keeps benchmark version, source model name, and available run details attached to the score.
The metric shown here is accuracy. It should be interpreted within SimpleQA Verified, not compared as part of a site-wide ranking.
What to be careful about
Scores depend on grading strictness; keep the source methodology attached.
Frequently asked
What is SimpleQA Verified?
A factual short-answer QA benchmark measuring parametric knowledge and hallucination resistance (Epoch AI's SimpleQA Verified). It is a other benchmark measured by accuracy.
What does accuracy mean on SimpleQA Verified?
SimpleQA Verified reports accuracy (%); higher is better. Scores are shown only within SimpleQA Verified and are never averaged with other benchmarks.
What is the top reported SimpleQA Verified score?
Gemini 3.1 Pro Preview has the top reported score on SimpleQA Verified: 77.3% (accuracy).
Why do SimpleQA Verified scores differ across runs?
Harness, scaffold, reasoning effort, and prompt setup change results, so two runs of the same model can differ. evals.report keeps each score with its run context so the differences stay visible.
Does evals.report rank models across benchmarks?
No. SimpleQA Verified scores are shown within their own metric; evals.report never combines benchmarks into a composite ranking or a single "best model".