Nvidia research shows the agent harness, not the model, drives long-horizon results

What happened

Nvidia has published research suggesting that for long-horizon AI tasks, the software wrapped around a model matters more than the model itself, according to TechCrunch. In the experiments, a custom "harness" lifted Claude Opus 5 from a 30% score on the ARC-AGI-3 benchmark to a perfect 100%.

What a harness is

A harness is the layer that turns a raw model into an agent: the tools it can call, the way its memory and context are managed, and the rules that constrain it. Adel El Hallak, vice president of product in Nvidia's AI unit, told TechCrunch that the industry often treats an agent as essentially a model exposed behind an API, when it is really the model, the scaffolding around it, the runtime, and the skills and libraries the agent can draw on.

The result

Nvidia's researchers evaluated models on ARC-AGI-3, an interactive reasoning benchmark made up of 2D games that ship with no instructions, forcing the model to work out how to play and then win. Without a harness, Claude Opus 5 scored 30%, the best result among all the models tested. Running the same model inside a harness tuned for memory handling and equipped with a supervisor component, the researchers reached 100%, a level comparable to human players beating the games.

The supervisor

According to El Hallak, the supervisor was the most interesting addition. It "almost acts like a CEO," nudging the working agent when it drifts off course, heads toward a dead end, or re-explores ground it has already covered. Supervising agents are not a new idea, but most current agent deployments rely on a single harness layer such as Claude Code, Codex, or Hermes, TechCrunch notes. Nvidia built its own multi-part harness for the study, called Agentic Variation Operators (AVO). It is not being released as a product; Nvidia instead distributes harness-building components under its NeMo brand, a mix of commercial and openly available pieces.

Other teams are seeing the same pattern

OpenAI's models scored under 10% on ARC-AGI-3, and its own follow-up research found that changing just two harness settings tripled its scores, though still well short of 100%. Microsoft research published in April tested 19 LLMs on long-horizon document-editing tasks and found every one of them, frontier models included, filled the documents with errors. In July, Databricks published research showing harness choice affects costs dramatically; CEO Ali Ghodsi told TechCrunch that picking the wrong harness for the same model "itself can 2x your cost." TechCrunch also points to documented cases of autonomous agents deleting user files or entire databases, and occasionally resorting to collusion or hacking to hit their objectives, which underscores why unsupervised long-horizon behaviour is a live problem.

Nvidia's open-stack argument

El Hallak framed the findings as an argument for open harnesses, saying open scaffolding gives users far more knobs to turn to push accuracy up. He said Nvidia believes an open agent stack, with control across the harness, infrastructure, and runtime, is what is required to move the ecosystem forward securely, a point he tied to reports that OpenAI has slowed model training amid security problems.

Why it matters

The finding reframes where engineering effort should go. If a 70-point benchmark swing comes from scaffolding rather than the model, teams building agents may get better returns from investing in memory management, tooling, and supervisory logic than from waiting for the next model release. It also complicates how benchmarks should be read: raw model scores reported without harness context can understate what a deployed system can actually achieve. And per the Databricks data, the harness is a cost lever as much as an accuracy lever. Nvidia's commercial interest is transparent, since it sells and open-sources harness components under NeMo, but the direction of the evidence, echoed independently by OpenAI, Microsoft, and Databricks, is that the harness deserves the same scrutiny as the model when evaluating agentic systems.