AA-Omniscience v1.1: Knowledge and Hallucination Benchmark
Artificial Analysis' revised factual-knowledge and hallucination benchmark: 3,000 private questions across six major domains, down from 6,000 in v1. Saturated and public-set questions were removed, new questions were added, and ambiguous or erroneous ones were revised. Questions are drawn from authoritative academic and industry sources by an LLM-based question-generation agent. The Net Score (-100 to 100) rewards correct answers, penalizes hallucinated ones, and does not penalize abstaining.
What is AA-Omniscience v1.1: Knowledge and Hallucination Benchmark?
Artificial Analysis' revised factual-knowledge and hallucination benchmark: 3,000 private questions across six major domains, down from 6,000 in v1. Saturated and public-set questions were removed, new questions were added, and ambiguous or erroneous ones were revised. Questions are drawn from authoritative academic and industry sources by an LLM-based question-generation agent. The Net Score (-100 to 100) rewards correct answers, penalizes hallucinated ones, and does not penalize abstaining. evals.report tracks reported AA-Omniscience v1.1: Knowledge and Hallucination Benchmark scores with the model, source, status, date, and run caveats attached — official leaderboard scores, vendor-reported launches, and clearly labeled community runs.
Top reported AA-Omniscience v1.1: Knowledge and Hallucination Benchmark score: Claude Fable 5.2 — 67 (AA-Omniscience Net Score).
| Model | Lab | Score↓ | Source model | Status | Date | |
|---|---|---|---|---|---|---|
| Claude Fable 5.2 | Anthropic | 67 | Claude Fable 5.2 (Max, Default Fallback) | Official | Nov 9, 2026 | Details |
| Gemini 4 Carbon | Google DeepMind | 54 | Gemini 4 Carbon (High) | Official | Nov 10, 2026 | Details |
| Claude Opus 5.5 | Anthropic | 48 | Claude Opus 5.5 (Max, Default Fallback) | Official | Sep 22, 2026 | Details |
| Gemini 4 Hydrogen | Google DeepMind | 46 | Gemini 4 Hydrogen (Xhigh) | Official | Nov 12, 2026 | Details |
| Gemini 4 Argon | Google DeepMind | 46 | Gemini 4 Argon (High) | Official | Oct 13, 2026 | Details |
| GPT-6 Astra | OpenAI | 45 | GPT-6 Astra (High) | Official | Sep 3, 2026 | Details |
| GPT-6.1 Sol | OpenAI | 42 | GPT-6.1 Sol (Xhigh) | Official | Sep 29, 2026 | Details |
| Grok 4.7 | xAI | 31 | Grok 4.7 (Xhigh) | Official | Sep 21, 2026 | Details |
| Muse Wonder Full | Meta | 31 | Muse Wonder Full (Max) | Official | Oct 12, 2026 | Details |
| Claude Sonnet 5.5 | Anthropic | 30 | Claude Sonnet 5.5 (Max, Default Fallback) | Official | Sep 28, 2026 | Details |
| Kimi K3.1 ProOpen | Moonshot AI | 30 | Kimi K3.1 Pro | Official | Oct 23, 2026 | Details |
| GPT-6.1 Terra | OpenAI | 30 | GPT-6.1 Terra (Max) | Official | Oct 26, 2026 | Details |
| Muse Spark OpenOpen | Meta | 26 | Muse Spark Open (Xhigh) | Official | Oct 15, 2026 | Details |
| Muse Wonder | Meta | 25 | Muse Wonder (Max) | Official | Oct 12, 2026 | Details |
| Gemini 3.8 Flash | Google DeepMind | 24 | Gemini 3.8 Flash (Medium) | Official | Sep 2, 2026 | Details |
| Step 5 Preview | StepFun | 16 | Step 5 Preview | Official | Sep 18, 2026 | Details |
| Inkling 2 SmallOpen | Thinking Machines Lab | 16 | Inkling 2 Small | Official | Nov 5, 2026 | Details |
| Claude Haiku 5.5 | Anthropic | 12 | Claude Haiku 5.5 (Max) | Official | Oct 7, 2026 | Details |
| Qwen3.8 Max (0902) | Alibaba / Qwen | 10 | Qwen3.8 Max (0902) | Official | Sep 2, 2026 | Details |
| Qwen3.8 Max | Alibaba / Qwen | 8 | Qwen3.8 2.4T A95B | Official | Aug 3, 2026 | Details |
| Muse Ember | Meta | 8 | Muse Ember (Xhigh) | Official | Nov 9, 2026 | Details |
| Gemini 3.5 Flash-Lite | Google DeepMind | 7 | Gemini 3.5 Flash-Lite | Official | Jul 21, 2026 | Details |
| GLM-5.3Open | Z.ai | 6 | GLM-5.3 (Max) | Official | Aug 14, 2026 | Details |
| Hy4 | Tencent | 5 | Tencent Hy4 | Official | Nov 4, 2026 | Details |
| Mistral Large 4Open | Mistral AI | 5 | Mistral Large 4 (High) | Official | Oct 29, 2026 | Details |
| NVIDIA Nemotron 3 UltraOpen | NVIDIA | 3 | Nemotron 3 Ultra 550B A55B (Reasoning) | Official | Jun 4, 2026 | Details |
| Beam 1.0Open | Reflection AI | 2 | Beam 1.0 | Official | Oct 16, 2026 | Details |
| DeepSeek V4.1 Flash 1017Open | DeepSeek | 2 | DeepSeek V4.1 Flash 1017 (Non-reasoning) | Official | Oct 17, 2026 | Details |
| JT-4.1 Flash 236B A21B | China Mobile | 2 | JT-4.1 Flash 236B A21B | Official | Nov 11, 2026 | Details |
| GLM-5.3-FlashOpen | Z.ai | 1 | GLM-5.3-Flash | Official | Aug 26, 2026 | Details |
| Ling 3.1 Flash 1019Open | inclusionAI (Ant Group) | 0 | Ling 3.1 Flash 1019 | Official | Oct 27, 2026 | Details |
| InklingOpen | Thinking Machines Lab | -2 | Inkling (Xhigh) | Official | Jul 15, 2026 | Details |
| NVIDIA Nemotron 3.5 Super VL 120B-A12BOpen | NVIDIA | -3 | Nemotron 3.5 Super VL | Official | Oct 15, 2026 | Details |
| Nanbeige4.3-33BOpen | Nanbeige LLM Lab | -4 | Nanbeige4.3-33B | Official | Oct 25, 2026 | Details |
| MiMo-V2.6-ProOpen | Xiaomi | -4 | MiMo-V2.6-Pro | Official | Sep 21, 2026 | Details |
| K2 Horizon 375B A23BOpen | MBZUAI | -5 | K2 Horizon 375B A23B | Official | Sep 3, 2026 | Details |
| MiniMax M3Open | MiniMax | -6 | MiniMax-M3 | Official | Jun 1, 2026 | Details |
| Qwen3.8-27BOpen | Alibaba / Qwen | -6 | Qwen3.8 27B (Xhigh) | Official | Aug 5, 2026 | Details |
| DeepSeek V4 Pro 0813Open | DeepSeek | -6 | DeepSeek V4 Pro 0813 (Max) | Official | Aug 13, 2026 | Details |
| MiniMax M3.1 FlashOpen | MiniMax | -6 | MiniMax-M3.1-Flash | Official | Oct 19, 2026 | Details |
| GPT-6 Luna | OpenAI | -8 | GPT-6 Luna (Max) | Official | Sep 22, 2026 | Details |
| Inkling SmallOpen | Thinking Machines Lab | -9 | Inkling Small | Official | Jul 30, 2026 | Details |
| MiMo-V2.6-FlashOpen | Xiaomi | -11 | MiMo-V2.6-Flash | Official | Sep 21, 2026 | Details |
| Agnes 2.5 Pro ThetaOpen | Sapiens AI | -11 | Agnes 2.5 Pro Theta | Official | Nov 7, 2026 | Details |
| Solar Mini 4 | Upstage | -12 | Solar Mini 4 | Official | Sep 22, 2026 | Details |
| NVIDIA Nemotron 3.5 Lightning 30B-A3BOpen | NVIDIA | -16 | Nemotron 3.5 Lightning | Official | Aug 11, 2026 | Details |
| Muse Glimmer 30BOpen | Meta | -32 | Muse Glimmer (High) | Official | Aug 10, 2026 | Details |
| Mistral Medium 3.5Open | Mistral AI | -39 | Mistral Medium 3.5 | Official | Apr 28, 2026 | Details |
| GPT-OSS-120BOpen | OpenAI | -46 | gpt-oss-120b (High) | Official | Aug 5, 2025 | Details |
Each row reports the model’s AA-Omniscience Net Score on AA-Omniscience v1.1: Knowledge and Hallucination Benchmark. Click a row for the full run context.