evals.report
BenchmarksLabsCompareRun guidesIn the wild
BenchmarksCoding

SWE-bench Pro

A harder public software-engineering agent benchmark built around professional repository tasks.

Coding% resolvedHigher is better
ScoresAboutRun this benchmark

What is SWE-bench Pro?

A harder public software-engineering agent benchmark built around professional repository tasks. evals.report tracks reported SWE-bench Pro scores with the model, source, status, date, and run caveats attached — official leaderboard scores, vendor-reported launches, and clearly labeled community runs.

Top reported SWE-bench Pro score: Claude Fable 5 — 80.0% (% resolved).

ModelLabScore↓Source modelStatusDate
Claude Fable 5Anthropic80.0%Claude Fable 5VerifiedJun 9, 2026Details
Claude Mythos PreviewAnthropic77.8%Claude Mythos PreviewUnverifiedApr 7, 2026Details
Fugu UltraSakana AI73.7%Fugu UltraVerifiedJun 15, 2026Details
Claude Opus 4.8Anthropic69.2%Claude Opus 4.8UnverifiedMay 28, 2026Details
Grok 4.5xAI64.7%Grok 4.5VerifiedJul 8, 2026Details
GPT-5.6 SolOpenAI64.6%GPT-5.6 SolVerifiedJul 9, 2026Details
GPT-5.6 TerraOpenAI63.4%GPT-5.6 TerraVerifiedJul 9, 2026Details
Claude Sonnet 5Anthropic63.2%Claude Sonnet 5UnverifiedJun 30, 2026Details
GPT-5.6 LunaOpenAI62.7%GPT-5.6 LunaVerifiedJul 9, 2026Details
Ornith-1.0-397BOpenDeepReinforce62.2%Ornith-1.0-397BVerifiedJun 25, 2026Details
GLM-5.2OpenZ.ai62.1%GLM-5.2VerifiedJun 16, 2026Details
Muse Spark 1.1Meta61.5%Muse Spark 1.1OfficialJul 9, 2026Details
GPT-5.4OpenAI59.10%gpt-5.4 (xHigh)*OfficialMar 5, 2026Details
MiniMax M3OpenMiniMax59.0%MiniMax M3VerifiedJun 1, 2026Details
FuguSakana AI59.0%FuguVerifiedJun 15, 2026Details
Nex-N2-ProOpenNex AGI58.8%Nex-N2-ProVerifiedJun 2, 2026Details
GPT-5.5OpenAI58.6%GPT-5.5VerifiedApr 23, 2026Details
MiMo-V2.5-ProOpenXiaomi57.2%MiMo-V2.5-ProVerifiedApr 22, 2026Details
Step 3.7 FlashOpenStepFun56.3%Step 3.7 FlashVerifiedMay 29, 2026Details
MiniMax M2.7OpenMiniMax56.22%MiniMax M2.7VerifiedMar 18, 2026Details
MiMo-V2.5OpenXiaomi56.1MiMo-V2.5VerifiedApr 22, 2026Details
DeepSeek V4 ProOpenDeepSeek55.4%DeepSeek V4 Pro MaxUnverifiedApr 24, 2026Details
MiniMax M2.5OpenMiniMax55.4%MiniMax-M2.5VerifiedFeb 12, 2026Details
Muse SparkMeta55.00%Muse Spark*OfficialApr 8, 2026Details
InklingOpenThinking Machines Lab54.3%InklingVerifiedJul 15, 2026Details
MAI-Thinking-1Microsoft AI52.8%MAI-Thinking-1VerifiedJun 2, 2026Details
DeepSeek V4 FlashOpenDeepSeek52.6%DeepSeek-V4-Flash (Think-Max)VerifiedApr 24, 2026Details
Claude Opus 4.6Anthropic51.90%claude-opus-4-6 (thinking)*OfficialFeb 5, 2026Details
MAI-Code-1-FlashMicrosoft AI51.2%MAI-Code-1-FlashUnverifiedJun 2, 2026Details
Nex-N2-miniOpenNex AGI50.2%Nex-N2-miniVerifiedJun 2, 2026Details
Doubao Seed 2.0 ProByteDance46.9%Seed2.0 Pro (0215)VerifiedFeb 14, 2026Details
Gemini 3.1 Pro PreviewGoogle DeepMind46.10%gemini-3.1-pro (thinking)*OfficialFeb 19, 2026Details
Claude Opus 4.5Anthropic45.89%claude-opus-4-5-20251101OfficialNov 24, 2025Details
Claude Sonnet 4.5Anthropic43.60%claude-4-5-SonnetOfficialSep 29, 2025Details
Gemini 3 ProGoogle DeepMind43.30%gemini-3-pro-previewOfficialNov 18, 2025Details
Claude Sonnet 4Anthropic42.70%claude-4-SonnetOfficialMay 22, 2025Details
GPT-5OpenAI41.78%gpt-5-2025-08-07 (High)OfficialAug 7, 2025Details
GPT-5.2-CodexOpenAI41.04%gpt-5.2-codexOfficialDec 18, 2025Details
Claude Haiku 4.5Anthropic39.45%claude-4-5-haikuOfficialOct 15, 2025Details
Qwen 3 Coder 480BOpenAlibaba / Qwen38.70%qwen3-coder-480b-a35bOfficialJul 22, 2025Details
MiniMax M2.1OpenMiniMax36.81%minimax-2.1OfficialDec 23, 2025Details
Gemini 3 FlashGoogle DeepMind34.63%gemini-3-flashOfficialDec 17, 2025Details
GPT-5.2OpenAI29.94%gpt-5.2OfficialDec 11, 2025Details
Kimi K2 InstructOpenMoonshot AI27.67%kimi-k2-instructOfficialJul 11, 2025Details
Qwen3 235B A22B Instruct 2507OpenAlibaba / Qwen21.41%qwen3-235b-a22bOfficialJul 21, 2025Details
GPT-OSS-120BOpenOpenAI16.20%gpt-oss-120bOfficialAug 5, 2025Details
DeepSeek V3.2OpenDeepSeek15.56%deepseek-v3p2OfficialDec 1, 2025Details
Llama 3.1 405BOpenMeta11.18%llama3-1-405b-instructOfficialJul 23, 2024Details
GLM-4.6OpenZ.ai9.67%glm-4.6OfficialSep 30, 2025Details
Llama 4 MaverickOpenMeta5.24%llama4-maverick-17b-instructOfficialApr 5, 2025Details

Each row reports the model’s % resolved on SWE-bench Pro. Click a row for the full run context.