evals.report
BenchmarksLabsCompareRun guidesIn the wild
OpenAIGPT

GPT-4o

OpenAI · GPT. Released May 13, 2024.

GPT-4o is a model from OpenAI in the GPT family, released May 13, 2024. evals.report tracks 43 reported GPT-4o benchmark scores across SWE-bench Verified, GPQA Diamond, FrontierMath, AIME (OTIS Mock), MMMU-Pro, LiveCodeBench Pro, Humanity's Last Exam, MCP Atlas, and 35 more — each shown with its benchmark, metric, source status, and date, and never combined into a single ranking.

43 results

Benchmark results 43

Compare this model
BenchmarkCategoryScoreMetricStatusDate
SWE-bench VerifiedCoding31.0%% resolvedOfficialMay 13, 2024Details
GPQA DiamondReasoning49.2%accuracyOfficialMay 13, 2024Details
FrontierMathReasoning0.34%accuracyOfficialMay 13, 2024Details
AIME (OTIS Mock)Reasoning6.4%accuracyOfficialMay 13, 2024Details
MMMU-ProMultimodal51.9%accuracyOfficialMay 13, 2024Details
LiveCodeBench ProCoding210Codeforces EloOfficialMay 13, 2024Details
Humanity's Last ExamReasoning2.7%accuracyOfficialMay 13, 2024Details
MCP AtlasTool use7.2%pass rateOfficialMay 13, 2024Details
Artificial Analysis Intelligence IndexReasoning17.3IndexUnverifiedMay 13, 2024Details
Epoch Capabilities IndexReasoning129.4IndexOfficialMay 13, 2024Details
Aider PolyglotCoding23.1%% correctOfficialMay 13, 2024Details
BrowseCompAgents0.6%accuracyVerifiedMay 13, 2024Details
τ²-bench (Telecom)Tool use23.5%pass^1UnverifiedMay 13, 2024Details
MathVistaMultimodal63.8%accuracyVerifiedMay 13, 2024Details
Video-MMEMultimodal71.9%accuracyOfficialMay 13, 2024Details
GDPvalAgents378EloOfficialMay 13, 2024Details
METR Task-Completion Time HorizonsAgents7.0 min50% time horizonOfficialMay 13, 2024Details
SciCodeCoding1.5%accuracyOfficialMay 13, 2024Details
MMMU (Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark)Multimodal69.1%accuracyVerifiedMay 13, 2024Details
LongBench v2Reasoning51.4%accuracyOfficialMay 13, 2024Details
Video-MMMUMultimodal61.2%accuracyOfficialMay 13, 2024Details
Search ArenaChat preference1006EloVerifiedMay 13, 2024Details
EQ-Bench Creative Writing v3Chat preference1484EloVerifiedMay 13, 2024Details
Design ArenaChat preference915EloVerifiedMay 13, 2024Details
AILuminate AI Safety BenchmarkOtherGoodSafety gradeVerifiedMay 13, 2024Details
MASK (Model Alignment between Statements and Knowledge)Other60.07Honesty scoreVerifiedMay 13, 2024Details
MCP-UniverseTool use15.58%Overall Success RateVerifiedMay 13, 2024Details
CharXivMultimodal58.8%accuracyUnverifiedMay 13, 2024Details
OCRBench v2Multimodal47.6accuracyVerifiedMay 13, 2024Details
FACTS GroundingReasoning78.8%Grounding accuracyVerifiedMay 13, 2024Details
BigCodeBenchCoding34.5%calibrated Pass@1VerifiedMay 13, 2024Details
SWE-bench MultimodalCoding30.37%% resolvedVerifiedMay 13, 2024Details
SuperGPQAReasoning44.40%accuracyVerifiedMay 13, 2024Details
EnigmaEvalReasoning0.80%accuracyVerifiedMay 13, 2024Details
ZeroBenchMultimodal0.0% (pass@1)accuracyVerifiedMay 13, 2024Details
PutnamBenchReasoning1Problems solvedVerifiedMay 13, 2024Details
Vectara Hallucination LeaderboardOther9.6%Hallucination RateOfficialMay 13, 2024Details
Gray Swan Arena (Agent Red-Teaming / Indirect Prompt Injection)Agents2.41%Attack Success Rate (ASR)VerifiedMay 13, 2024Details
PolyMath: Evaluating Mathematical Reasoning in Multilingual ContextsReasoning13.7Difficulty-Weighted Accuracy (DW-ACC)VerifiedMay 13, 2024Details
Online-Mind2WebAgents30.7%Task success rateOfficialMay 13, 2024Details
WebArenaAgents42.8%Task success rateVerifiedMay 13, 2024Details
GSO: Software Optimization Benchmark for SWE-AgentsCoding0.0%Opt@1OfficialMay 13, 2024Details
MultiNRCReasoning12.42%accuracyOfficialMay 13, 2024Details