GPT-6 Astra controls robot arms: 19/20 on block pick-and-place, but stalls on precision insertion

A frontier model on physical hardware

A benchmark published on September 4 by RoboCurve, which rose to the Hacker News front page, handed OpenAI's GPT-6 Astra control of a physical bimanual robot rig and compared it against Anthropic's Claude Fable 5 and Fable 5.1 on two tabletop manipulation tasks. The outcome is a clear split: Astra was dramatically better at coarse pick-and-place, while on a precision insertion task its advantage disappeared entirely.

The report is a follow-up to the team's earlier Fable 5 versus Fable 5.1 comparison. Astra ran the identical setup: the same YAM arms, the same Inspect Robots agent policy, and the same two tasks, with 20 trials per model per task and 120 counted runs in total.

The bowl task

Task one was simple pick-and-place: lift a red block off a table and put it inside a bowl. According to the report, Astra completed it in 19 of 20 trials, against 8 of 20 for Fable 5.1 and 1 of 20 for Fable 5. Astra's runs were also faster, about 2.5 minutes per trial versus 6.8 for Fable 5.1, and cheaper at an estimated $0.94 per run against $2.12. The report's summary charts frame that as a 2.4× higher completion rate at roughly 2.3× lower cost.

The puzzle task

Task two demanded fine motor control: pick up a round blue puzzle piece by a knob at its center and seat it in the matching circular groove of a board. Here Astra completed the insertion just 2 times in 20 trials — identical to Fable 5.1's result. The report notes that Astra reaches the groove and then stalls at the same final step that trips up the Fable models. It still cost less per attempt, $1.36 versus $2.18, about 1.6× cheaper, but the capability gap vanished.

How the evaluation worked

The hardware was a pair of I2RT YAM arms, six degrees of freedom per arm with parallel-jaw grippers. The model does not send joint commands directly; it issues absolute end-effector poses — position, orientation and gripper state, per arm — and the robot's inverse kinematics converts them to joint angles. Each turn the model sees three camera views (top-down plus left and right wrist cameras) alongside proprioceptive state.

The agent policy capped each trial at 20 LLM calls, ran at medium thinking effort with a 25% speed cap, and kept default safety guardrails on. The harness was Inspect Robots 0.58.0. Every trial was scored by a human grader on the furthest of five stages it reached: no purposeful approach, contact with the object, lifted clear of the table, positioned above the deposit point, and finally placed. Costs were computed from wire-level request and response tokens at list price, $10 and $50 per million input and output tokens for all three models.

Caveats the authors flag

The report is unusually candid about its weaknesses:

• Astra's trials ran two days after the Fable trials and were not interleaved with them.
• The puzzle comparison used the same rig, but the headline bowl comparison did not: the Fable models ran on rig-3, which was unavailable when Astra was tested on rig-1.
• Grading was done by the operator with the model's identity known, leaving room for unconscious bias.
• Cost comparisons are not perfectly symmetric: Anthropic requests were sent without prompt caching, while OpenAI automatically cached about a fifth of Astra's input. Since that discount is not reflected in the figures, the authors say Astra's cost is, if anything, overstated.
• Objects were reset by hand between trials, and every model ran at medium reasoning effort only.

Why it matters

Public head-to-head tests of frontier language models on real robot hardware are rare, and this one offers two practical signals. First, general-purpose pick-and-place appears to be edging toward reliability: near-perfect block-into-bowl at around a dollar and two and a half minutes per run is a usable capability, not a lab demo. Second, the failure pattern is telling — the best-performing model stalls at exactly the same final insertion step as its rivals, which suggests precision alignment is a shared bottleneck in language-model control of robots rather than a weakness specific to one vendor.

The caveats matter: the headline result comes from different rigs, the grading was not blind, and the trials were not interleaved, so the exact ratios deserve some skepticism. Directionally, though, the report is a data point that frontier models are becoming credible general-purpose robot brains for coarse manipulation, while fine-grained insertion remains an open problem across the board.