SciCode
A scientist-curated benchmark that evaluates language models on realistic scientific research coding problems, comprising 338 subproblems decomposed from 80 challenging main problems across 16 natural-science subfields (physics, math, chemistry, biology, materials science).
What this benchmark measures
A scientist-curated benchmark that evaluates language models on realistic scientific research coding problems, comprising 338 subproblems decomposed from 80 challenging main problems across 16 natural-science subfields (physics, math, chemistry, biology, materials science).
Rows on this page are sourced from public benchmark artifacts, leaderboard exports, or source-linked model reports. Each row keeps benchmark version, source model name, and available run details attached to the score.
The metric shown here is accuracy. It should be interpreted within SciCode, not compared as part of a site-wide ranking.
Frequently asked
What is SciCode?
A scientist-curated benchmark that evaluates language models on realistic scientific research coding problems, comprising 338 subproblems decomposed from 80 challenging main problems across 16 natural-science subfields (physics, math, chemistry, biology, materials science). It is a coding benchmark measured by accuracy.
What does accuracy mean on SciCode?
SciCode reports accuracy (%); higher is better. Scores are shown only within SciCode and are never averaged with other benchmarks.
What is the top reported SciCode score?
Fugu has the top reported score on SciCode: 60.1% (accuracy).
Why do SciCode scores differ across runs?
Harness, scaffold, reasoning effort, and prompt setup change results, so two runs of the same model can differ. evals.report keeps each score with its run context so the differences stay visible.
Does evals.report rank models across benchmarks?
No. SciCode scores are shown within their own metric; evals.report never combines benchmarks into a composite ranking or a single "best model".