OpenAI-MRCR v2 (Multi-Round Coreference Resolution)
A long-context retrieval benchmark in which a model must locate and reproduce a specific instance (the i-th 'needle') of repeated similar requests buried in a long synthetic multi-turn conversation, scored on the 8-needle variant across context lengths up to 1M tokens.
What this benchmark measures
A long-context retrieval benchmark in which a model must locate and reproduce a specific instance (the i-th 'needle') of repeated similar requests buried in a long synthetic multi-turn conversation, scored on the 8-needle variant across context lengths up to 1M tokens.
Rows on this page are sourced from public benchmark artifacts, leaderboard exports, or source-linked model reports. Each row keeps benchmark version, source model name, and available run details attached to the score.
The metric shown here is accuracy (mean SequenceMatcher similarity). It should be interpreted within OpenAI-MRCR v2 (Multi-Round Coreference Resolution), not compared as part of a site-wide ranking.
Frequently asked
What is OpenAI-MRCR v2 (Multi-Round Coreference Resolution)?
A long-context retrieval benchmark in which a model must locate and reproduce a specific instance (the i-th 'needle') of repeated similar requests buried in a long synthetic multi-turn conversation, scored on the 8-needle variant across context lengths up to 1M tokens. It is a reasoning benchmark measured by accuracy (mean SequenceMatcher similarity).
What does accuracy (mean SequenceMatcher similarity) mean on OpenAI-MRCR v2 (Multi-Round Coreference Resolution)?
OpenAI-MRCR v2 (Multi-Round Coreference Resolution) reports accuracy (mean SequenceMatcher similarity) (%); higher is better. Scores are shown only within OpenAI-MRCR v2 (Multi-Round Coreference Resolution) and are never averaged with other benchmarks.
What is the top reported OpenAI-MRCR v2 (Multi-Round Coreference Resolution) score?
Claude Opus 4.6 has the top reported score on OpenAI-MRCR v2 (Multi-Round Coreference Resolution): 93.0% (accuracy (mean SequenceMatcher similarity)).
Why do OpenAI-MRCR v2 (Multi-Round Coreference Resolution) scores differ across runs?
Harness, scaffold, reasoning effort, and prompt setup change results, so two runs of the same model can differ. evals.report keeps each score with its run context so the differences stay visible.
Does evals.report rank models across benchmarks?
No. OpenAI-MRCR v2 (Multi-Round Coreference Resolution) scores are shown within their own metric; evals.report never combines benchmarks into a composite ranking or a single "best model".