evals.report
BenchmarksLabsCompareRun guidesIn the wild
Models
1 selected
o3OpenAI
Benchmarks
3 selected
SWE-bench VerifiedCodingDeepSWECodingSWE-bench ProCoding
Benchmarko3OpenAI
SWE-bench Verified% resolved62.3%
DeepSWE% resolved—
SWE-bench Pro% resolved—
62.3%
% resolved
—
% resolved
—

No aggregate score is calculated. Each row uses its benchmark’s own metric. Compare rows independently.