evals.report
BenchmarksLabsCompareRun guidesIn the wild
Google DeepMindGemini

Gemini 1.5 Pro

Google DeepMind · Gemini. Released Feb 15, 2024.

Gemini 1.5 Pro is a model from Google DeepMind in the Gemini family, released Feb 15, 2024. evals.report tracks 14 reported Gemini 1.5 Pro benchmark scores across GPQA Diamond, AIME (OTIS Mock), Artificial Analysis Intelligence Index, Epoch Capabilities Index, MathVista, Video-MME, SciCode, MMMU (Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark), and 6 more — each shown with its benchmark, metric, source status, and date, and never combined into a single ranking.

14 results

Benchmark results 14

Compare this model
BenchmarkCategoryScoreMetricStatusDate
GPQA DiamondReasoning57.2%accuracyOfficialFeb 15, 2024Details
AIME (OTIS Mock)Reasoning23.1%accuracyOfficialFeb 15, 2024Details
Artificial Analysis Intelligence IndexReasoning16IndexUnverifiedFeb 15, 2024Details
Epoch Capabilities IndexReasoning132.8IndexOfficialFeb 15, 2024Details
MathVistaMultimodal63.9%accuracyVerifiedFeb 15, 2024Details
Video-MMEMultimodal75.0%accuracyOfficialFeb 15, 2024Details
SciCodeCoding1.5%accuracyOfficialFeb 15, 2024Details
MMMU (Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark)Multimodal65.9%accuracyUnverifiedFeb 15, 2024Details
Video-MMMUMultimodal53.9%accuracyOfficialFeb 15, 2024Details
AILuminate AI Safety BenchmarkOtherGoodSafety gradeVerifiedFeb 15, 2024Details
OCRBench v2Multimodal51.6accuracyVerifiedFeb 15, 2024Details
FACTS GroundingReasoning80.0%Grounding accuracyVerifiedFeb 15, 2024Details
BigCodeBenchCoding32.4%calibrated Pass@1VerifiedFeb 15, 2024Details
ZeroBenchMultimodal0.0% (pass@1)accuracyVerifiedFeb 15, 2024Details