evals.report
BenchmarksLabsCompareRun guidesIn the wild
BenchmarksMultimodal

CharXiv

A multimodal benchmark of 2,323 real scientific charts from arXiv papers that evaluates chart understanding in MLLMs via descriptive questions and complex reasoning questions, with the reasoning split (CharXiv-R) measuring accuracy on questions that require synthesizing information across chart elements.

MultimodalaccuracyHigher is better
ScoresAboutRun this benchmark

What is CharXiv?

A multimodal benchmark of 2,323 real scientific charts from arXiv papers that evaluates chart understanding in MLLMs via descriptive questions and complex reasoning questions, with the reasoning split (CharXiv-R) measuring accuracy on questions that require synthesizing information across chart elements. evals.report tracks reported CharXiv scores with the model, source, status, date, and run caveats attached — official leaderboard scores, vendor-reported launches, and clearly labeled community runs.

Top reported CharXiv score: Claude Mythos Preview — 93.2% (accuracy).

ModelLabScore↓Source modelStatusDate
Claude Mythos PreviewAnthropic93.2%—UnverifiedApr 7, 2026Details
Claude Opus 4.7Anthropic91.0%—UnverifiedApr 16, 2026Details
Claude Opus 4.8Anthropic89.9%—UnverifiedMay 28, 2026Details
Kimi K2.6OpenMoonshot AI86.7%—UnverifiedApr 20, 2026Details
Fugu UltraSakana AI86.6%Fugu UltraVerifiedJun 15, 2026Details
Muse SparkMeta86.4%—UnverifiedApr 8, 2026Details
FuguSakana AI85.1%FuguVerifiedJun 15, 2026Details
Gemini 3.5 FlashGoogle DeepMind84.2%—UnverifiedMay 19, 2026Details
GPT-5.5OpenAI84.1%—UnverifiedApr 23, 2026Details
GPT-5.2OpenAI82.1%—UnverifiedDec 11, 2025Details
Qwen 3.6 PlusAlibaba / Qwen81.5%—UnverifiedApr 2, 2026Details
Gemini 3 ProGoogle DeepMind81.4%—UnverifiedNov 18, 2025Details
GPT-5OpenAI81.1%—UnverifiedAug 7, 2025Details
Gemini 3 FlashGoogle DeepMind80.3%—UnverifiedDec 17, 2025Details
o3OpenAI78.6%—UnverifiedApr 16, 2025Details
InklingOpenThinking Machines Lab78.1%InklingVerifiedJul 15, 2026Details
Kimi K2.5OpenMoonshot AI77.5%—UnverifiedJan 27, 2026Details
Claude Opus 4.6Anthropic77.4%—UnverifiedFeb 5, 2026Details
o4-miniOpenAI72.0%—UnverifiedApr 16, 2025Details
GPT-4oOpenAI58.8%—UnverifiedMay 13, 2024Details
GPT-4.1OpenAI56.7%—UnverifiedApr 14, 2025Details

Each row reports the model’s accuracy on CharXiv. Click a row for the full run context.