evals.report
BenchmarksLabsCompareRun guidesIn the wild

FrontierCode 2.0

Cognition's second-generation code-mergeability benchmark (released 2026-10-27): 132 private tasks crafted by the open-source maintainers of the repositories they come from, graded against maintainer-defined rubrics where any failed blocker criterion scores the solution 0. Compared with FrontierCode 1.1, tasks come from 55% more repositories and add categories such as security patches and performance optimizations; reference solutions touch 67% more files and 89% more lines in codebases 25% larger; and tasks carry 43% more blocking criteria, with a new emphasis on 'thoroughness' (fixing every instance of a bug, matching existing support for related inputs). In 80% of tasks the reference solution was corrected from the upstream PR. Each model is run 5× at every available reasoning effort; runs that consulted solution-bearing sources are zeroed.

Codingweighted score (Main)Higher is better

No run guide for this benchmark yet.