evals.report
BenchmarksLabsCompareRun guidesIn the wild
Models
4 selected
GPT-5.5OpenAIClaude Opus 4.8AnthropicGemini 3.1 Pro PreviewGoogle DeepMindDeepSeek V4 ProDeepSeek
Benchmarks
3 selected
SWE-bench VerifiedCodingDeepSWECodingSWE-bench ProCoding
BenchmarkGPT-5.5OpenAIClaude Opus 4.8AnthropicGemini 3.1 Pro PreviewGoogle DeepMindDeepSeek V4 ProOpenDeepSeek
SWE-bench Verified% resolved80.6%88.6%75.6%80.6%
DeepSWE% resolved70.05%58%9.88%7.52%
SWE-bench Pro% resolved58.6%69.2%46.10%55.4%

No aggregate score is calculated. Each row uses its benchmark’s own metric. Compare rows independently.