AA-Omniscience v1.1: Knowledge and Hallucination Benchmark
Artificial Analysis' revised factual-knowledge and hallucination benchmark: 3,000 private questions across six major domains, down from 6,000 in v1. Saturated and public-set questions were removed, new questions were added, and ambiguous or erroneous ones were revised. Questions are drawn from authoritative academic and industry sources by an LLM-based question-generation agent. The Net Score (-100 to 100) rewards correct answers, penalizes hallucinated ones, and does not penalize abstaining.
What this benchmark measures
Artificial Analysis' revised factual-knowledge and hallucination benchmark: 3,000 private questions across six major domains, down from 6,000 in v1. Saturated and public-set questions were removed, new questions were added, and ambiguous or erroneous ones were revised. Questions are drawn from authoritative academic and industry sources by an LLM-based question-generation agent. The Net Score (-100 to 100) rewards correct answers, penalizes hallucinated ones, and does not penalize abstaining.
Rows on this page are sourced from public benchmark artifacts, leaderboard exports, or source-linked model reports. Each row keeps benchmark version, source model name, and available run details attached to the score.
The metric shown here is AA-Omniscience Net Score. It should be interpreted within AA-Omniscience v1.1: Knowledge and Hallucination Benchmark, not compared as part of a site-wide ranking.
What to be careful about
Not comparable with the v1 column (aa-omniscience). AA notes that removing the saturated questions lowers most models' scores, and that this reflects the dataset change rather than a model regression. Some models still move up: Claude Opus 5.5 goes from 46 on v1 to 48 on v1.1. Each model appears once, at its highest-scoring setting on the board.
Frequently asked
What is AA-Omniscience v1.1: Knowledge and Hallucination Benchmark?
Artificial Analysis' revised factual-knowledge and hallucination benchmark: 3,000 private questions across six major domains, down from 6,000 in v1. Saturated and public-set questions were removed, new questions were added, and ambiguous or erroneous ones were revised. Questions are drawn from authoritative academic and industry sources by an LLM-based question-generation agent. The Net Score (-100 to 100) rewards correct answers, penalizes hallucinated ones, and does not penalize abstaining. It is a reasoning benchmark measured by AA-Omniscience Net Score.
What does AA-Omniscience Net Score mean on AA-Omniscience v1.1: Knowledge and Hallucination Benchmark?
AA-Omniscience v1.1: Knowledge and Hallucination Benchmark reports AA-Omniscience Net Score; higher is better. Scores are shown only within AA-Omniscience v1.1: Knowledge and Hallucination Benchmark and are never averaged with other benchmarks.
What is the top reported AA-Omniscience v1.1: Knowledge and Hallucination Benchmark score?
Claude Fable 5.2 has the top reported score on AA-Omniscience v1.1: Knowledge and Hallucination Benchmark: 67 (AA-Omniscience Net Score).
Why do AA-Omniscience v1.1: Knowledge and Hallucination Benchmark scores differ across runs?
Harness, scaffold, reasoning effort, and prompt setup change results, so two runs of the same model can differ. evals.report keeps each score with its run context so the differences stay visible.
Does evals.report rank models across benchmarks?
No. AA-Omniscience v1.1: Knowledge and Hallucination Benchmark scores are shown within their own metric; evals.report never combines benchmarks into a composite ranking or a single "best model".