MCP Atlas
Scale AI's large-scale tool-use benchmark: 1,000 expert-written natural-language tasks over 36 real Model Context Protocol (MCP) servers and 220+ tools, requiring agents to discover and orchestrate multi-step tool calls; scored by pass rate via an LLM judge.
What this benchmark measures
Scale AI's large-scale tool-use benchmark: 1,000 expert-written natural-language tasks over 36 real Model Context Protocol (MCP) servers and 220+ tools, requiring agents to discover and orchestrate multi-step tool calls; scored by pass rate via an LLM judge.
Rows on this page are sourced from public benchmark artifacts, leaderboard exports, or source-linked model reports. Each row keeps benchmark version, source model name, and available run details attached to the score.
The metric shown here is pass rate. It should be interpreted within MCP Atlas, not compared as part of a site-wide ranking.
What to be careful about
Pass rate uses an LLM judge (default Gemini 2.5 Pro); MiniMax's number is a self-reported Public Set run, distinct from Scale's official leaderboard.
Frequently asked
What is MCP Atlas?
Scale AI's large-scale tool-use benchmark: 1,000 expert-written natural-language tasks over 36 real Model Context Protocol (MCP) servers and 220+ tools, requiring agents to discover and orchestrate multi-step tool calls; scored by pass rate via an LLM judge. It is a tool use benchmark measured by pass rate.
What does pass rate mean on MCP Atlas?
MCP Atlas reports pass rate (%); higher is better. Scores are shown only within MCP Atlas and are never averaged with other benchmarks.
What is the top reported MCP Atlas score?
Muse Spark 1.1 has the top reported score on MCP Atlas: 88.1% (pass rate).
Why do MCP Atlas scores differ across runs?
Harness, scaffold, reasoning effort, and prompt setup change results, so two runs of the same model can differ. evals.report keeps each score with its run context so the differences stay visible.
Does evals.report rank models across benchmarks?
No. MCP Atlas scores are shown within their own metric; evals.report never combines benchmarks into a composite ranking or a single "best model".