Anthropic's 90% multi-agent research win carries a 15x token bill

What Anthropic measured

Anthropic built a research system in which a lead agent parcels out work to subagents that run in parallel. According to figures summarised in a dev.to post by Mark Fulton, the setup beat a single-agent baseline by 90.2% on the company's internal research evaluation.

The same write-up put a price on the win: the multi-agent system consumed roughly fifteen times as many tokens as an ordinary chat, and token usage by itself explained 80% of the variance in performance on the BrowseComp benchmark.

Put side by side, the two numbers change the reading of the result. The multi-agent system did not prevail because several models reason better together; it prevailed largely because the task allowed far more compute to be spent in parallel on pieces that did not depend on each other. The deciding question, as the dev.to analysis frames it, is whether the job actually splits.

Where the design fits, and where it does not

Anthropic's write-up names the poor fits: domains where every agent needs the same context, and tasks with heavy dependencies between agents. Most coding work has fewer genuinely parallel pieces than research does, according to the write-up.

Research is the ideal case: ten sources can be read by ten agents that never interact. A module refactor behaves more like a pipeline — if subtask B needs subtask A's output before starting, spreading the work across five agents mostly adds handoffs where context can be lost.

How these systems fail

A separate study published in 2025 by researchers from UC Berkeley and collaborators, "Why Do Multi-Agent LLM Systems Fail?", collected more than 1,600 annotated traces across seven popular multi-agent frameworks. With an inter-annotator agreement of 0.88 kappa, the team identified fourteen failure modes in three categories:

• System design issues, roughly four failures in ten: agents ignoring the task specification, repeating steps, and lacking a clear stopping condition.
• Inter-agent misalignment, about a third: the largest single mode is a mismatch between an agent's stated reasoning and its actions, followed by failures to ask for clarification and drifting off task.
• Task verification, about a quarter: no verification, wrong verification, or work treated as finished before it is.

The conspicuous omission is model capability. The authors state plainly that failure is not simply a function of the underlying model's limits; most of what goes wrong is structural — who owns what, what counts as done, and who checks the result.

What helps, and what does not

The researchers tested remediations on one framework. Clearer role specifications lifted the success rate by 9.4%, and adding a high-level verification step lifted it by 15.6%. Real improvements, but the authors still concluded that isolated patches are not enough and that reliability requires deeper redesign.

Drawing on both sets of findings, the dev.to author proposes four tests before splitting one agent into several: confirm the work genuinely splits rather than forming a pipeline; price the job against the fifteenfold token bill; write an explicit brief for each agent covering its input, output format and stop condition; and name a checker, human or agent, that verifies output against the spec before anything ships.

Why it matters

Anthropic's figures make the trade unusually concrete: a 90.2% performance gain is real, but so is a token bill around fifteen times that of a plain chat, and much of the measured gain tracks raw token spend rather than architectural sophistication. For engineering teams, that turns the multi-agent question into a cost decision as much as a technical one — as the dev.to author argues, a routine task run a hundred times a day rarely justifies the multiplier.

The failure data points the same way. Because most failures are structural — vague briefs, missing termination conditions, absent verification — they will survive the next model upgrade, since they live in the design rather than the weights. The scarce skill is shifting from choosing a model to designing roles, briefs and checks: less prompt writing, more management.