CharXiv
A multimodal benchmark of 2,323 real scientific charts from arXiv papers that evaluates chart understanding in MLLMs via descriptive questions and complex reasoning questions, with the reasoning split (CharXiv-R) measuring accuracy on questions that require synthesizing information across chart elements.
What this benchmark measures
A multimodal benchmark of 2,323 real scientific charts from arXiv papers that evaluates chart understanding in MLLMs via descriptive questions and complex reasoning questions, with the reasoning split (CharXiv-R) measuring accuracy on questions that require synthesizing information across chart elements.
Rows on this page are sourced from public benchmark artifacts, leaderboard exports, or source-linked model reports. Each row keeps benchmark version, source model name, and available run details attached to the score.
The metric shown here is accuracy. It should be interpreted within CharXiv, not compared as part of a site-wide ranking.
Frequently asked
What is CharXiv?
A multimodal benchmark of 2,323 real scientific charts from arXiv papers that evaluates chart understanding in MLLMs via descriptive questions and complex reasoning questions, with the reasoning split (CharXiv-R) measuring accuracy on questions that require synthesizing information across chart elements. It is a multimodal benchmark measured by accuracy.
What does accuracy mean on CharXiv?
CharXiv reports accuracy (%); higher is better. Scores are shown only within CharXiv and are never averaged with other benchmarks.
What is the top reported CharXiv score?
Claude Mythos Preview has the top reported score on CharXiv: 93.2% (accuracy).
Why do CharXiv scores differ across runs?
Harness, scaffold, reasoning effort, and prompt setup change results, so two runs of the same model can differ. evals.report keeps each score with its run context so the differences stay visible.
Does evals.report rank models across benchmarks?
No. CharXiv scores are shown within their own metric; evals.report never combines benchmarks into a composite ranking or a single "best model".