evals.report
BenchmarksLabsCompareRun guidesIn the wild
BenchmarksCoding

SWE-bench Verified

A curated SWE-bench split for evaluating systems that resolve real software engineering issues.

Coding% resolvedHigher is better
ScoresAboutRun this benchmark

What is SWE-bench Verified?

A curated SWE-bench split for evaluating systems that resolve real software engineering issues. evals.report tracks reported SWE-bench Verified scores with the model, source, status, date, and run caveats attached — official leaderboard scores, vendor-reported launches, and clearly labeled community runs.

Top reported SWE-bench Verified score: Claude Fable 5 — 95.0% (% resolved).

ModelLabScore↓Source modelStatusDate
Claude Fable 5Anthropic95.0%Claude Fable 5VerifiedJun 9, 2026Details
Claude Mythos PreviewAnthropic93.9%Claude Mythos PreviewUnverifiedApr 7, 2026Details
Claude Opus 4.8Anthropic88.6%Claude Opus 4.8VerifiedMay 28, 2026Details
Claude Opus 4.7Anthropic83.5%Claude Opus 4.7OfficialApr 16, 2026Details
Ornith-1.0-397BOpenDeepReinforce82.4%Ornith-1.0-397BVerifiedJun 25, 2026Details
Nex-N2-ProOpenNex AGI80.8%Nex-N2-ProVerifiedJun 2, 2026Details
GPT-5.5OpenAI80.6%GPT-5.5OfficialApr 23, 2026Details
DeepSeek V4 ProOpenDeepSeek80.6%DeepSeek V4 Pro MaxUnverifiedApr 24, 2026Details
MiniMax M2.5OpenMiniMax80.2%MiniMax-M2.5VerifiedFeb 12, 2026Details
DeepSeek V4 FlashOpenDeepSeek79.0%DeepSeek-V4-Flash (Think-Max)VerifiedApr 24, 2026Details
MiMo-V2.5-ProOpenXiaomi78.9%MiMo-V2.5-ProVerifiedApr 22, 2026Details
Claude Opus 4.6Anthropic78.7%Claude Opus 4.6OfficialFeb 5, 2026Details
MiniMax M2.7OpenMiniMax78%MiniMax M2.7UnverifiedMar 18, 2026Details
InklingOpenThinking Machines Lab77.6%InklingVerifiedJul 15, 2026Details
Mistral Medium 3.5Mistral AI77.6%Mistral Medium 3.5VerifiedApr 28, 2026Details
GPT-5.4OpenAI76.9%GPT-5.4OfficialMar 5, 2026Details
Kimi K2.6OpenMoonshot AI76.7%Kimi K2.6OfficialApr 20, 2026Details
Claude Opus 4.5Anthropic76.7%Claude Opus 4.5OfficialNov 24, 2025Details
Qwen 3.6 Max PreviewAlibaba / Qwen76.7%Qwen 3.6 Max (Preview)OfficialApr 20, 2026Details
Doubao Seed 2.0 ProByteDance76.5%Seed 2.0 ProVerifiedFeb 14, 2026Details
Doubao Seed 2.0 CodeByteDance76.5%Seed 2.0 Code (shares Pro coding benchmarks)UnverifiedFeb 14, 2026Details
Step 3.7 FlashOpenStepFun76.5%Step 3.7 FlashVerifiedMay 29, 2026Details
Qwen3.5-397B-A17BOpenAlibaba / Qwen76.4%Qwen3.5-397B-A17BUnverifiedFeb 16, 2026Details
Gemini 3.1 Pro PreviewGoogle DeepMind75.6%Gemini 3.1 ProOfficialFeb 19, 2026Details
Gemini 3 FlashGoogle DeepMind75.4%Gemini 3 FlashOfficialDec 17, 2025Details
Claude Sonnet 4.6Anthropic75.2%Claude Sonnet 4.6OfficialFeb 17, 2026Details
GPT-5.3-CodexOpenAI74.8%GPT-5.3 CodexOfficialFeb 5, 2026Details
Nex-N2-miniOpenNex AGI74.4%Nex-N2-miniVerifiedJun 2, 2026Details
GLM-5.1OpenZ.ai74.2%GLM-5.1OfficialApr 7, 2026Details
Kimi K2.5OpenMoonshot AI73.8%Kimi K2.5OfficialJan 27, 2026Details
GPT-5.2OpenAI73.8%GPT-5.2OfficialDec 11, 2025Details
GLM-4.7OpenZ.ai73.8%GLM-4.7VerifiedDec 22, 2025Details
GPT-5OpenAI73.6%GPT-5OfficialAug 7, 2025Details
MAI-Thinking-1Microsoft AI73.5%MAI-Thinking-1VerifiedJun 2, 2026Details
Claude Opus 4.1Anthropic73.3%Claude Opus 4.1OfficialAug 5, 2025Details
Gemini 3 ProGoogle DeepMind72.9%Gemini 3 ProOfficialNov 18, 2025Details
GLM-5OpenZ.ai72.1%GLM-5OfficialFeb 11, 2026Details
NVIDIA Nemotron 3 UltraOpenNVIDIA71.9%Nemotron-3-Ultra-550B-A55B (BF16)VerifiedJun 4, 2026Details
MAI-Code-1-FlashMicrosoft AI71.6%MAI-Code-1-FlashUnverifiedJun 2, 2026Details
Claude Sonnet 4.5Anthropic71.3%Claude Sonnet 4.5OfficialSep 29, 2025Details
Kimi K2 ThinkingOpenMoonshot AI71.3%Kimi K2 ThinkingVerifiedNov 6, 2025Details
Claude Opus 4Anthropic70.7%Claude Opus 4OfficialMay 22, 2025Details
Amazon Nova 2 ProAmazon70.0%Nova 2 ProVerifiedDec 2, 2025Details
GPT-5.1OpenAI68.0%GPT-5.1OfficialNov 12, 2025Details
DeepSeek V3.1OpenDeepSeek66.0%DeepSeek-V3.1VerifiedAug 21, 2025Details
GPT-5 miniOpenAI64.7%GPT-5 miniOfficialAug 7, 2025Details
Amazon Nova 2 LiteAmazon64.5%Nova 2 LiteVerifiedDec 2, 2025Details
o3OpenAI62.3%o3OfficialApr 16, 2025Details
Claude 3.7 SonnetAnthropic61.0%Claude 3.7 SonnetOfficialFeb 24, 2025Details
NVIDIA Nemotron 3 Super 120B-A12BOpenNVIDIA60.47%N-3-SuperVerifiedMar 10, 2026Details
Qwen 3.6 PlusAlibaba / Qwen57.9%Qwen 3.6 PlusOfficialApr 2, 2026Details
Gemini 2.5 ProGoogle DeepMind57.6%Gemini 2.5 Pro (Jun 2025)OfficialMar 25, 2025Details
K-EXAONEOpenLG AI Research49.4%K-EXAONE (Reasoning)UnverifiedJan 12, 2026Details
GPT-4.1OpenAI48.5%GPT-4.1OfficialApr 14, 2025Details
Solar Pro 2Upstage40.8%Solar Pro 2 [R, Reasoning]VerifiedJul 10, 2025Details
GPT-4oOpenAI31.0%GPT-4oOfficialMay 13, 2024Details

Each row reports the model’s % resolved on SWE-bench Verified. Click a row for the full run context.