GPT-6 Astra catches up to 20% more cross-file bugs in early CodeRabbit review

What CodeRabbit measured

CodeRabbit, in an evaluation that reached Hacker News' front page, tested OpenAI's GPT-6 Astra against the models it already uses for automated code review. The metric is "actionable bug coverage": how many labeled bugs a model surfaces through findings a developer can actually act on.

The headline numbers are modest. Astra caught roughly 4% more labeled bugs than GPT-5.6 Sol and 22% more than Opus 5. CodeRabbit attributes the small overall gap to the evaluation mix, which includes simpler reviews where a stronger model has little room to stand out. The company explicitly labels the result early and directional.

Gains concentrate in cross-file reviews

The more encouraging subset is the harder one: cross-file reviews, where a change looks correct in isolation but breaks something elsewhere in the codebase. There, Astra's relative advantage grows to 20% over Sol and 33% over Opus 5.

CodeRabbit's interpretation is that Astra's main advance lies in connecting the right pieces of information when relevant evidence is distributed across a repository. The post is careful about cause and effect: it does not claim that a larger context window alone improves performance, that the results rank overall review quality, or that the same gain will appear on every pull request.

The price of stronger reasoning

Astra's published standard API rates are $10 per million input tokens and $50 per million output tokens. Fable 5.1 lists the same base rates, though its caching prices differ. Using an illustrative task of 100,000 uncached input tokens and 10,000 billable output tokens (reasoning tokens included), CodeRabbit calculates that Astra costs 2.5 times Sol, roughly 4.7 times Terra, and about 47 times Luna at that fixed usage. The comparison excludes caching, cache writes, tools, retries, regional uplifts and service-tier adjustments; prices were checked September 4, 2026, and Sol's promotional pricing runs at least through November 21, 2026.

Those multiples are not predictions of cost per completed task. CodeRabbit notes that OpenAI reports lower estimated task costs for Astra in some of its own evaluations despite the higher token prices, since a model that needs fewer tokens or fewer attempts can narrow the gap. The practical advice is to measure total cost per successful outcome on your own workload rather than deciding on token price or a capability benchmark alone. CodeRabbit also frames running the model at customer scale as requiring protection of customer data alongside the pricing assessment.

Where the skill might transfer

Beyond review, CodeRabbit sees the transferable capability as reasoning over relationships among separate sources. It lists candidate uses it has not measured: reconciling conflicting research reports, assembling incident explanations from logs and runbooks while separating observations from hypotheses, tracing a proposed change across specifications and internal policies, and checking whether formulas, assumptions and narrative conclusions agree across a report and its supporting files. The shared structure is dispersed evidence with dependencies between its parts.

A whole game as a stress test

CodeRabbit also used Astra to build NIGHTSHIFT, an action RPG written in Godot and GDScript. The scope is large: seven character classes, a 988-node passive skill tree inspired by Path of Exile, active skills, runes, skill evolutions, and co-op play across 40 zones in 10 acts. The team says the hardest problem was rebalancing interconnected systems after core changes mid-development, and that Astra handled platform work autonomously, including native PS5 and Xbox controller support, macOS, web and Linux builds, and the Xcode project, certificates and notarization needed to distribute LAN co-op builds. The game was also built so agents themselves can play, with Astra joining live co-op sessions as a teammate.

Why it matters

Code review is one of the first places where better model reasoning has a measurable, dollar-attached consequence: bugs caught before merge versus tokens billed per review. CodeRabbit's numbers suggest the newest frontier models earn their premium mainly on hard, cross-file work rather than routine checks, which points teams toward routing difficult tasks to expensive models and leaving easy ones to cheaper options. The open question is whether per-task efficiency offsets the token-price premium, and the post argues that can only be answered by running the new model alongside the current one and comparing quality, verification time and total cost.