GPT-5.6 Luna vs GPT-6 Astra: a $0.004 review finds 75% of the bugs a $0.11 review does

A benchmark from Entelligence, currently circulating on the Hacker News front page, puts concrete numbers on a question many engineering teams now face: is the cheapest frontier model good enough to review pull requests? Across 50 deliberately bugged PRs, a single review by GPT-5.6 Luna cost $0.0041 while GPT-6 Astra cost $0.113 — a 28x gap per review — yet Luna still caught three quarters as many verified bugs as its far pricier sibling.

How the test was run

Entelligence drew on 50 public benchmark pull requests from the AI-Code-Review-Evals organization: ten each from Cal.com, Sentry, Discourse, Keycloak and Grafana, with defects deliberately introduced against clean base branches. Both models received the same prompt on the same diffs, asking for correctness, security, concurrency, resource and error-handling issues while excluding style, naming, documentation and test suggestions.

Verification pooled findings from Astra, Luna, GPT-5.6 Sol and public Entelligence reviewer comments into one anonymized list, which Astra and Sol then judged independently. A finding only counts as verified when both judges call it real. They agreed on 91% of findings, and 143 distinct bugs passed both. On pricing, Luna charges $0.20 per million input tokens and $1.20 per million output tokens; Astra charges $10 and $50.

The headline numbers

• Verified bugs: 69 for Luna, 92 for Astra
• Total findings raised: 93 versus 96
• Precision: 74% versus 96% — 24 of Luna's findings failed verification, against 4 of Astra's
• Total cost across 50 PRs: $0.20 versus $5.66
• Cost per verified bug: $0.0030 versus $0.061
• Mean review time: 23 seconds versus 36 seconds

Luna produced roughly 3.1x more output tokens per review (2,104 versus 688) and still ended up far cheaper, because its output price is 42x lower. According to Entelligence, the takeaway is that Luna is adequate for everyday correctness bugs at that price, but the authors would not trust it alone on authentication or permission code.

Where the cheap model breaks down

The aggregate score hides large per-repository gaps. In Sentry, Discourse and Grafana, Luna landed within two verified bugs of Astra. Cal.com was wider at 21 to 30. Keycloak, an identity and access management server, was worst: Luna found 6 verified bugs to Astra's 14, and only half its Keycloak findings survived verification, against 93% for Astra.

The bug-class split points the same way. Luna found 39 data and logic bugs to Astra's 47, and 10 of 13 concurrency bugs. On security it found 9 of 24 where Astra found 19. Two Keycloak misses illustrate the difference: federated recovery codes that were never marked as used, permitting reuse, and a global view permission that silently trumped denials set on individual clients. Neither is visible on a single line — spotting them requires working out what the permission model allows after the change.

Luna also caught things Astra missed. Of the 143 verified bugs, 44 came from both models, 48 only from Astra and 25 only from Luna, mostly data and logic or concurrency issues. Running both models on every PR would have surfaced 117 bugs, or 82%, for $5.86 in total — Luna's $0.20 on top of Astra's $5.66 buys 25 extra verified findings.

Caveats the authors flagged

• Each model reviewed each PR once, apart from a small repeat test on ten PRs. Astra reproduced 10 of its 15 verified bugs in both reruns; Luna reproduced 7 of 15. Single-run numbers are therefore rough, especially for Luna.
• Astra is both a contestant and one of the two judges, which Entelligence acknowledges could tilt results slightly in its favor, even with Sol required to agree.
• The benchmark PRs date from 2013 to July 25, 2025, all before either model's training cutoff, so the date-split contamination test readers requested is impossible here. The defects were added specifically for the benchmark, but models familiar with the correct upstream code may still hold an edge.
• Both models saw only the diff — no repository history, call graph or production data.
• At least 26 verified bugs were caught only by Sol or the Entelligence reviewer, meaning both compared models miss real issues, and the true count is higher because unnoticed bugs never enter the pool.

Why it matters

This is unusually actionable data for model selection economics. A 28x per-review gap and a 20x per-verified-bug gap give teams a concrete basis for routing: cheap models for routine correctness checks, premium ones for anything touching auth or permissions, or both in tandem, where roughly $0.004 per PR buys 25 additional findings. The precision cost is the other side of the ledger — when one comment in four is wrong, developers learn to skim, and that erodes trust in the entire review pipeline. The benchmark also shows the ceiling of diff-only review: every model missed bugs that only surface with wider context.