evals.report
BenchmarksLabsCompareRun guidesIn the wild
DeepSeekDeepSeek

DeepSeek V3

DeepSeek · DeepSeek. Released Dec 26, 2024.

DeepSeek V3 is a model from DeepSeek in the DeepSeek family, released Dec 26, 2024. evals.report tracks 15 reported DeepSeek V3 benchmark scores across GPQA Diamond, FrontierMath, AIME (OTIS Mock), LMArena, Artificial Analysis Intelligence Index, Epoch Capabilities Index, MMLU-Pro, GAIA: A Benchmark for General AI Assistants, and 7 more — each shown with its benchmark, metric, source status, and date, and never combined into a single ranking.

Open15 results

Benchmark results 15

Compare this model
BenchmarkCategoryScoreMetricStatusDate
GPQA DiamondReasoning56.5%accuracyOfficialDec 26, 2024Details
FrontierMathReasoning1.72%accuracyOfficialDec 26, 2024Details
AIME (OTIS Mock)Reasoning15.8%accuracyOfficialDec 26, 2024Details
LMArenaChat preference1332source-defined ratingOfficialDec 26, 2024Details
Artificial Analysis Intelligence IndexReasoning16.5IndexUnverifiedDec 26, 2024Details
Epoch Capabilities IndexReasoning133.1IndexOfficialDec 26, 2024Details
MMLU-ProReasoning75.9%accuracyVerifiedDec 26, 2024Details
GAIA: A Benchmark for General AI AssistantsAgents29.39%accuracyUnverifiedDec 26, 2024Details
MCP-UniverseTool use14.29%Overall Success RateVerifiedDec 26, 2024Details
BigCodeBenchCoding40.5%calibrated Pass@1VerifiedDec 26, 2024Details
SuperGPQAReasoning47.40%accuracyVerifiedDec 26, 2024Details
IMO-BenchReasoning37.0%accuracyVerifiedDec 26, 2024Details
Vectara Hallucination LeaderboardOther6.1%Hallucination RateOfficialDec 26, 2024Details
PolyMath: Evaluating Mathematical Reasoning in Multilingual ContextsReasoning20.4Difficulty-Weighted Accuracy (DW-ACC)VerifiedDec 26, 2024Details
Online-Mind2WebAgents32.33%Task success rateVerifiedDec 26, 2024Details