MultiChallenge
A realistic multi-turn conversation benchmark by Scale AI (SEAL) that evaluates whether frontier LLMs can follow instructions, retain user information, perform versioned editing, and stay self-coherent across multiple conversational turns.
What this benchmark measures
A realistic multi-turn conversation benchmark by Scale AI (SEAL) that evaluates whether frontier LLMs can follow instructions, retain user information, perform versioned editing, and stay self-coherent across multiple conversational turns.
Rows on this page are sourced from public benchmark artifacts, leaderboard exports, or source-linked model reports. Each row keeps benchmark version, source model name, and available run details attached to the score.
The metric shown here is accuracy. It should be interpreted within MultiChallenge, not compared as part of a site-wide ranking.
Frequently asked
What is MultiChallenge?
A realistic multi-turn conversation benchmark by Scale AI (SEAL) that evaluates whether frontier LLMs can follow instructions, retain user information, perform versioned editing, and stay self-coherent across multiple conversational turns. It is a reasoning benchmark measured by accuracy.
What does accuracy mean on MultiChallenge?
MultiChallenge reports accuracy (%); higher is better. Scores are shown only within MultiChallenge and are never averaged with other benchmarks.
What is the top reported MultiChallenge score?
Muse Spark has the top reported score on MultiChallenge: 75.52% (accuracy).
Why do MultiChallenge scores differ across runs?
Harness, scaffold, reasoning effort, and prompt setup change results, so two runs of the same model can differ. evals.report keeps each score with its run context so the differences stay visible.
Does evals.report rank models across benchmarks?
No. MultiChallenge scores are shown within their own metric; evals.report never combines benchmarks into a composite ranking or a single "best model".