AILuminate AI Safety Benchmark
MLCommons' standardized AI safety benchmark that grades how often general-purpose chat models produce policy-violating responses across 12 hazard categories (e.g. violent crimes, CSAM, hate, self-harm, specialized advice), assigning an ordinal safety grade from Poor to Excellent relative to a sub-15B open-weight reference system.
What this benchmark measures
MLCommons' standardized AI safety benchmark that grades how often general-purpose chat models produce policy-violating responses across 12 hazard categories (e.g. violent crimes, CSAM, hate, self-harm, specialized advice), assigning an ordinal safety grade from Poor to Excellent relative to a sub-15B open-weight reference system.
Rows on this page are sourced from public benchmark artifacts, leaderboard exports, or source-linked model reports. Each row keeps benchmark version, source model name, and available run details attached to the score.
The metric shown here is Safety grade. It should be interpreted within AILuminate AI Safety Benchmark, not compared as part of a site-wide ranking.
Frequently asked
What is AILuminate AI Safety Benchmark?
MLCommons' standardized AI safety benchmark that grades how often general-purpose chat models produce policy-violating responses across 12 hazard categories (e.g. violent crimes, CSAM, hate, self-harm, specialized advice), assigning an ordinal safety grade from Poor to Excellent relative to a sub-15B open-weight reference system. It is a other benchmark measured by Safety grade.
What does Safety grade mean on AILuminate AI Safety Benchmark?
AILuminate AI Safety Benchmark reports Safety grade; higher is better. Scores are shown only within AILuminate AI Safety Benchmark and are never averaged with other benchmarks.
What is the top reported AILuminate AI Safety Benchmark score?
Claude 3.5 Sonnet has the top reported score on AILuminate AI Safety Benchmark: Very Good (Safety grade).
Why do AILuminate AI Safety Benchmark scores differ across runs?
Harness, scaffold, reasoning effort, and prompt setup change results, so two runs of the same model can differ. evals.report keeps each score with its run context so the differences stay visible.
Does evals.report rank models across benchmarks?
No. AILuminate AI Safety Benchmark scores are shown within their own metric; evals.report never combines benchmarks into a composite ranking or a single "best model".