Vectara Hallucination Leaderboard
Measures how often LLMs introduce hallucinations when summarizing short documents, scored by Vectara's HHEM-2.3 factual-consistency model, reported as a hallucination rate.
What this benchmark measures
Measures how often LLMs introduce hallucinations when summarizing short documents, scored by Vectara's HHEM-2.3 factual-consistency model, reported as a hallucination rate.
Rows on this page are sourced from public benchmark artifacts, leaderboard exports, or source-linked model reports. Each row keeps benchmark version, source model name, and available run details attached to the score.
The metric shown here is Hallucination Rate. It should be interpreted within Vectara Hallucination Leaderboard, not compared as part of a site-wide ranking.
Frequently asked
What is Vectara Hallucination Leaderboard?
Measures how often LLMs introduce hallucinations when summarizing short documents, scored by Vectara's HHEM-2.3 factual-consistency model, reported as a hallucination rate. It is a other benchmark measured by Hallucination Rate.
What does Hallucination Rate mean on Vectara Hallucination Leaderboard?
Vectara Hallucination Leaderboard reports Hallucination Rate (%); lower is better. Scores are shown only within Vectara Hallucination Leaderboard and are never averaged with other benchmarks.
What is the top reported Vectara Hallucination Leaderboard score?
Mistral Large has the top reported score on Vectara Hallucination Leaderboard: 4.5% (Hallucination Rate).
Why do Vectara Hallucination Leaderboard scores differ across runs?
Harness, scaffold, reasoning effort, and prompt setup change results, so two runs of the same model can differ. evals.report keeps each score with its run context so the differences stay visible.
Does evals.report rank models across benchmarks?
No. Vectara Hallucination Leaderboard scores are shown within their own metric; evals.report never combines benchmarks into a composite ranking or a single "best model".