OpenAI-MRCR v2 (Multi-Round Coreference Resolution)

Name: OpenAI-MRCR v2 (Multi-Round Coreference Resolution)
Creator: evals.report
License: https://creativecommons.org/licenses/by/4.0/

A long-context retrieval benchmark in which a model must locate and reproduce a specific instance (the i-th 'needle') of repeated similar requests buried in a long synthetic multi-turn conversation, scored on the 8-needle variant across context lengths up to 1M tokens.

Reasoningaccuracy (mean SequenceMatcher similarity)Higher is better

Scores About Run this benchmark

No run guide for this benchmark yet.