GPT-6 Astra hits 99.9% on ARC-AGI-3 as harness choice swings score and cost

ARC Prize has published its evaluation of OpenAI's GPT-6 Astra on ARC-AGI-3, the third generation of its abstract reasoning benchmark, and the model set state-of-the-art marks under both harnesses the organisation tested. The headline numbers point in two directions at once: 62.7% for $26,098 with the Standard harness at maximum reasoning effort, and 99.9% for $18,817 with the Provider Adapter harness at high effort. The post, which surfaced on the Hacker News front page, frames the gap between those two runs as part of the story.

What ARC-AGI-3 tests

ARC-AGI-3 places agents in novel, turn-based environments built from core knowledge priors, with no explicit instructions. Agents have to explore, infer goals from sparse rewards, and build internal models of each environment to plan effectively. The benchmark evaluates four components: exploration, modeling, goal-setting, and planning and execution. Difficulty was calibrated through controlled testing with human participants, and humans can solve 100% of the environments. The series' stated purpose is to measure the "residual gap" between current AI and AGI, which ARC Prize defines as a system's ability to acquire any skill a human can, as efficiently as a human can.

The harness changes everything

The two harnesses differ in how much context survives between turns. The Standard harness lets the model carry forward notes it chooses to keep. The Provider Adapter preserves opaque reasoning state between requests and compacts longer conversations, so earlier reasoning can be reused.

Under the Standard harness, scores scale with reasoning effort, from 35.2% at $49,791 with reasoning disabled up to 62.7% at $26,098 at max. Counterintuitively, thinking harder cost less: according to ARC Prize, higher effort settings solved games in fewer actions, which reduced the total number of model calls and tokens.

The Provider Adapter flattened those differences. Every effort level from none to max landed between 96.7% and 99.9%. The single best score, 99.9%, came at high effort for $18,817, while the cheapest run, max effort, reached 98.6% for $17,332. ARC Prize describes both configurations as state-of-the-art.

Ahead of the median human on action efficiency

Before launching the benchmark, ARC Prize tested roughly 500 members of the general public to establish an action-efficiency baseline, defined per level as the median action count among players who completed it. In the Provider Adapter harness at max effort, Astra used fewer actions than that baseline on 96.0% of levels and averaged 51.7% fewer actions per level, which ARC Prize calls a material milestone and a pass of human parity on this measure.

The organisation notes it had hypothesised that action efficiency would remain a dividing line between humans and AI. That still holds for brute-force approaches, but frontier models show what ARC Prize describes as a more binary pattern: once the model understands the mechanics, execution generally falls within the range of human efficiency.

For cost context, human participants were paid $115 per 90-minute session plus $5 per completed game, working out to about $12.78 per attempted game. Pricing only the brain's energy as electricity, ARC Prize estimates roughly 0.6 cents per session, or 0.067 cents per game attempted.

An improvised algebra

Astra's replays show it converting unfamiliar game mechanics into compact symbolic world models. It tracked objects, coordinates, rules and unfinished plans, and generated its own domain-specific shorthand notation for the environments. ARC Prize's published examples include operations mapped to the coordinates of the controls that perform them, ordered multi-step plans recorded as terse edit sequences, and turn counters combined with a player's position and orientation. Similar note-taking behaviour has appeared in other models, but ARC Prize says Astra's notes stood out for their precision and information density, distilling a scene into a code-like model of what exists, how the pieces interact, and which actions must happen in what order.

Why it matters

A roughly 37-point score swing produced by test infrastructure alone is a sharp reminder that frontier benchmark numbers cannot be read without their harness context; the scaffolding around a model is now as consequential as the model settings themselves. The results also invert the usual compute intuition, with the strongest reasoning settings costing less because they finish tasks sooner. Most significantly for the benchmark's own framing, surpassing the human baseline on action efficiency narrows a gap ARC Prize specifically built the test to expose. The caveats are worth holding onto: these scores are on the semi-private evaluation set, humans still solve 100% of the environments, and a near-ceiling result on one benchmark measures one slice of what the series defines as general intelligence.