Tests find GPT-6 Astra max reasoning effort costs 2.3x low for identical answers

Max costs 2.3x low for the same answers

A benchmark published on dev.to by Synthorai, the company behind a model-comparison site, ran GPT-6 Astra across its reasoning_effort ladder and found that the most expensive setting buys nothing on tasks the cheapest useful rung already solves.

The reasoning_effort parameter controls how much hidden thinking the model performs before answering, and that thinking is billed as reasoning tokens at the output rate. The author assembled 11 tasks — among them a rule applied 40 times in a row, a counting problem under three constraints, a knapsack instance, a base conversion and 7 to the power of 222 modulo 1000 — and solved each by exhaustive search on a local machine first, so the answer key was settled before any API call. Each task then ran three times at every effort setting through an endpoint that speaks the OpenAI Chat Completions API.

The result was a cliff, not a slope. Every setting from low through max scored a perfect 33 of 33 on both GPT-6 Astra and GPT-5.6 Sol. The only value that changed accuracy was none, which failed 17 of 33 runs on Astra and returned three different wrong numbers across three attempts at one iterated-map task. According to the post, there is nothing gradual to tune against: either reasoning is on and the answers are right, or it is off and the model is guessing.

Cost, meanwhile, scales with the setting. On the five hardest tasks, max averaged 370 reasoning tokens per call at $0.01946, against 151 tokens and $0.00851 at low — 2.3 times the price for identical, fully correct answers, with reasoning tokens metered at the $50-per-million output rate.

The accepted values contradict the docs

The post documents a three-way disagreement over which values the parameter takes. The model page lists five rungs — low, medium, high, xhigh and max — and says Astra does not support none. The API's first validation check, which screens request shape before any model is involved, advertises seven values in its error text: none, minimal, low, medium, high, xhigh and max. A second, model-specific check then rejects minimal on both Astra and Sol — an advertised value that works nowhere — while accepting none on both models despite the documentation.

One value appears in no list at all: disabled, accepted on Astra but rejected on Sol. Far from switching reasoning off, it behaves like a mid-ladder rung, averaging 243 reasoning tokens against low's 151 and costing 54% more per correct answer at identical accuracy. The error messages also reveal the dated build behind the gpt-6-astra alias: gpt-6-astra-2026-09-03.

Launch scores come from the best rung

The post ties this pricing to OpenAI's launch benchmarks, which it says are footnoted as the maximum at any effort — each published number comes from whichever setting scored best, and on the independent Artificial Analysis leaderboard that best entry is Astra's max, one point above xhigh. The current v4.2 of that index, as cited in the post, places Claude Fable 5.1 first at 57 and Astra third at 55.

OpenAI's own launch table, reproduced in the post, puts Astra far ahead on agent-style and mathematical work: 57.9% on Terminal-Bench 4.0 against 37.3% for GPT-5.6 Sol, 64.6% on Terminal-Bench Science, 97.6% on FrontierMath Tier 4 and 99.9% on ARC-AGI-3. It trails Claude Fable 5.1 on Humanity's Last Exam with tools, 57.2% to 65.0%, and on the Artificial Analysis Intelligence Index — gaps the post notes appear in the table but not in the launch prose. The cybersecurity figures, it adds, were produced, in OpenAI's words, "without production safeguards", and the shipping model will refuse proof-of-concept exploit tasks.

Cost per correct answer versus GPT-5.6 Sol

At list price — $10 in and $50 out per million tokens for Astra against $4 and $20 for Sol, as shown on the OpenAI model pages on 2026-09-07 — the sticker gap is 2.5x. Measured per correct answer across the 11 tasks, where both models were perfect from low upward, the real ratio is 1.57x at low ($0.00560 against $0.00356), 1.66x at medium, 1.85x at high, 2.27x at xhigh and 2.57x at max.

Why it matters

For API users this is a direct cost lever: on work the model already gets right at low effort, running max multiplies the bill by roughly 2.3 for no measurable gain. The post's advice is to measure your own tasks at low first and climb the ladder only where accuracy actually improves.

It is also a reminder that headline scores are not cost-normalized. A benchmark reported at the maximum of any effort setting describes a model's ceiling, not its value, and hides that one model's ceiling may cost far more to reach than another's. The undocumented disabled value that spends more than low is a further trap for anyone building configuration interfaces or cost models against the documentation.

One caveat: this is a single self-published benchmark from a company that sells comparison tooling, so treat the numbers as a prompt to test your own workload rather than a settled result.