Opus 5.5's cheaper cache reads upend one developer's text-to-SQL cost math

What happened

A developer writing on dev.to has redone the cost calculations for a text-to-SQL assistant in the wake of Claude Opus 5.5, and concluded that a single pricing line — cheaper prompt cache reads — is enough to overturn an architectural decision. According to the post, cache reads on the new model cost $0.20 per million tokens, down 60% from $0.50 on Opus 5, while input drops from $5 to $4, output from $25 to $20, and a five-minute cache write from $6.25 to $5.

The author's assistant, called Aria, lets CRM agents ask questions in plain English and get answers from live SQL. To keep prompts small, it runs retrieval over documentation of the schema rather than the data: roughly 90 plain-English descriptions of tables, columns and enum values are embedded and searched with pgvector, and only the five most relevant documents, around 1,100 tokens, go to the model with each question. The full semantic layer is about 20,000 tokens.

The redone math

The post compares two designs for the schema portion of the prompt. Option A sends only the top five retrieved documents, which cannot be cached because they change with every question. Option B sends the entire semantic layer on every request as a cached prefix.

Using the author's assumptions — 800 questions a day from 40 agents, and about 10 cache writes a day because the five-minute cache stays warm through the workday — the results shift sharply between model generations. On Opus 5, the full-schema approach cost $9.25 per day against $4.40 for retrieval, a 2.1x gap. On Opus 5.5, that becomes $4.20 against $3.52, or 1.2x. Per question the author calculates $0.0044 versus $0.0053, working out to roughly $77 versus $92 over a 22-day month.

At that margin, the author argues, cost no longer settles the question. Accuracy does — and it was never measured. The post notes real downsides to retrieval: an extra network hop before every question, and the risk that an embedding search misses the one document the query depends on. The author's example is a question about "stale leads" failing to surface the doc explaining the last_activity_at column, which makes the SQL wrong before generation even starts. A hybrid is the likely end state: a static cached schema plus retrieved example question-and-SQL pairs promoted from user feedback.

Two breaking changes in the migration guide

The post also flags two changes in the Opus 5.5 migration guide that land directly on text-to-SQL setups.

First, tool_choice values of tool or any now return a 400 error. Many such systems force the model to call a run_sql tool so it cannot answer from memory; the recommended fix is tool_choice: auto combined with marking the tool as strict and stating in the prompt when it must be used. The author keeps existing safeguards — SELECT-only validation, agent identity injected from the JWT, and a read-only Postgres role — unchanged, on the reasoning that upgrading the model should not shift where trust is placed.

Second, conversations are expected to be append-only. Thinking is always enabled in Opus 5.5 and, per the guide, replaying a thinking block after an edit to the system prompt, tools, or earlier messages returns a 400 by default for newer accounts. That conflicts with Aria's current design, which rebuilds the system prompt each turn with a fresh set of retrieved schema documents. The post offers two compliant paths: move retrieved docs into each new user turn, or make the system prompt static — which is the full-schema option again.

Why it matters

This is one developer's back-of-envelope analysis, not a benchmark, and the author is explicit about that. But the direction it captures is broader: pricing and API constraints are pushing toward the same architecture, in which large, rarely changing context sits in a static cached prefix and new context arrives only by appending. If cache reads keep falling relative to input prices, retrieval layers built purely to save tokens lose their main justification, and what remains is an empirical question — whether a model writes better SQL when shown five relevant tables or all fifteen. The author plans to answer it with a 30-example evaluation comparing row-level results, latency and reported cache-hit usage between the two designs, and promises to publish the numbers.