BenchmarksOther
SimpleQA Verified
A factual short-answer QA benchmark measuring parametric knowledge and hallucination resistance (Epoch AI's SimpleQA Verified).
OtheraccuracyHigher is better
What is SimpleQA Verified?
A factual short-answer QA benchmark measuring parametric knowledge and hallucination resistance (Epoch AI's SimpleQA Verified). evals.report tracks reported SimpleQA Verified scores with the model, source, status, date, and run caveats attached — official leaderboard scores, vendor-reported launches, and clearly labeled community runs.
Top reported SimpleQA Verified score: Gemini 3.1 Pro Preview — 77.3% (accuracy).
| Model | Lab | Score↓ | Source model | Status | Date | |
|---|---|---|---|---|---|---|
| Gemini 3.1 Pro Preview | Google DeepMind | 77.3% | Gemini 3.1 Pro | Official | Feb 19, 2026 | Details |
| Gemini 3 Pro | Google DeepMind | 72.9% | Gemini 3 Pro | Official | Nov 18, 2025 | Details |
| Gemini 3.5 Flash | Google DeepMind | 68.4% | Gemini 3.5 Flash | Official | May 19, 2026 | Details |
| Qwen3 Max | Alibaba / Qwen | 67.5% | Qwen3-Max | Official | Sep 5, 2025 | Details |
| Gemini 3 Flash | Google DeepMind | 67.4% | Gemini 3 Flash | Official | Dec 17, 2025 | Details |
| Muse Spark | Meta | 66.3% | Muse Spark | Official | Apr 8, 2026 | Details |
| GPT-5.5 Pro | OpenAI | 64.5% | GPT-5.5 Pro | Official | Apr 23, 2026 | Details |
| GPT-5.5 | OpenAI | 63.1% | GPT-5.5 | Official | Apr 23, 2026 | Details |
| Qwen 3.6 Max Preview | Alibaba / Qwen | 56.9% | Qwen 3.6 Max (Preview) | Official | Apr 20, 2026 | Details |
| Gemini 2.5 Pro | Google DeepMind | 56.0% | Gemini 2.5 Pro (Jun 2025) | Official | Mar 25, 2025 | Details |
| o3 | OpenAI | 53.0% | o3 | Official | Apr 16, 2025 | Details |
| Claude Opus 4.7 | Anthropic | 50.6% | Claude Opus 4.7 | Official | Apr 16, 2026 | Details |
| GPT-5 | OpenAI | 50.6% | GPT-5 | Official | Aug 7, 2025 | Details |
| Qwen3 235B A22B Instruct 2507Open | Alibaba / Qwen | 50.1% | Qwen3-235B-A22B (Jul 2025) | Official | Jul 21, 2025 | Details |
| Qwen 3.6 Plus | Alibaba / Qwen | 49.1% | Qwen 3.6 Plus | Official | Apr 2, 2026 | Details |
| GPT-5.1 | OpenAI | 48.9% | GPT-5.1 | Official | Nov 12, 2025 | Details |
| Grok 4 | xAI | 47.9% | Grok 4 | Official | Jul 9, 2025 | Details |
| GPT-5.4 Pro | OpenAI | 47.8% | GPT-5.4 Pro | Official | Mar 5, 2026 | Details |
| Claude Opus 4.6 | Anthropic | 46.5% | Claude Opus 4.6 | Official | Feb 5, 2026 | Details |
| GPT-5.4 | OpenAI | 44.8% | GPT-5.4 | Official | Mar 5, 2026 | Details |
| InklingOpen | Thinking Machines Lab | 43.9% | Inkling | Verified | Jul 15, 2026 | Details |
| Claude Opus 4.5 | Anthropic | 41.8% | Claude Opus 4.5 | Official | Nov 24, 2025 | Details |
| GPT-5.2 | OpenAI | 38.9% | GPT-5.2 | Official | Dec 11, 2025 | Details |
| Kimi K2.6Open | Moonshot AI | 38.7% | Kimi K2.6 | Official | Apr 20, 2026 | Details |
| GLM-5.1Open | Z.ai | 37.3% | GLM-5.1 | Official | Apr 7, 2026 | Details |
| Claude Opus 4.1 | Anthropic | 34.8% | Claude Opus 4.1 | Official | Aug 5, 2025 | Details |
| DeepSeek V4 FlashOpen | DeepSeek | 34.1% | DeepSeek-V4-Flash (Think-Max) | Verified | Apr 24, 2026 | Details |
| Kimi K2.5Open | Moonshot AI | 33.9% | Kimi K2.5 | Official | Jan 27, 2026 | Details |
| MAI-Thinking-1 | Microsoft AI | 31% | MAI-Thinking-1 | Verified | Jun 2, 2026 | Details |
| Claude Sonnet 4.6 | Anthropic | 29.0% | Claude Sonnet 4.6 | Official | Feb 17, 2026 | Details |
| DeepSeek V3.2Open | DeepSeek | 27.5% | DeepSeek-V3.2 | Official | Dec 1, 2025 | Details |
| o4-mini | OpenAI | 23.9% | o4-mini | Official | Apr 16, 2025 | Details |
| Claude Sonnet 4.5 | Anthropic | 23.6% | Claude Sonnet 4.5 | Official | Sep 29, 2025 | Details |
| GPT-5 mini | OpenAI | 21.0% | GPT-5 mini | Official | Aug 7, 2025 | Details |
| GPT-OSS-120BOpen | OpenAI | 13.9% | gpt-oss-120b | Official | Aug 5, 2025 | Details |
| Claude Haiku 4.5 | Anthropic | 5.9% | Claude Haiku 4.5 | Official | Oct 15, 2025 | Details |
Each row reports the model’s accuracy on SimpleQA Verified. Click a row for the full run context.