Prompt caching wrote 12,000 tokens and read zero, raising an agent's bill 25%

Enabling prompt caching on an LLM agent loop raised its bill by roughly a quarter instead of cutting it, because every request wrote about 12,000 tokens to the cache and read back exactly zero. That is the story in a post-mortem published on dev.to, where the author traces the silent failure to a timestamp injected by a logging helper written months earlier, and lays out the mental model needed to keep it from happening again.

What the usage object revealed

The agent loop resends a 12,000-token system prompt on every turn, and input tokens dominate the bill in a tool loop, so caching looked like an obvious win. Instead, nothing appeared broken: every request returned 200, latency stayed flat, and only three fields in the usage object told the truth — 12,184 tokens billed as cache creation, zero as cache reads, and 291 plain input tokens. The author paid the write premium roughly 40 times in a row and never collected a discounted read.

The pricing is what turns a misconfiguration into an active regression. According to the post, cache writes cost 1.25 times the base input price (twice that for the one-hour TTL), while reads cost about a tenth. Two requests that genuinely share a prefix come out ahead — 1.35 times base input together versus 2.0 uncached — but a write that is never read is pure overhead, and forty of them become a line item.

A prefix match, not a key-value store

The core mental model: the cache is not keyed on your system prompt. The API renders the request as tools, then system, then messages, hashes the bytes up to each cache_control breakpoint, and looks for an existing prefix. The first differing byte ends the match, and everything downstream is cold.

The post lists three ways this bites:

• A dynamic header, such as a current-time string placed in front of an 11,000-token stable playbook.
• A nondeterministic serializer — .dumps without sort_keys produces different key order, and therefore different bytes, for the same dict.
• A per-user tool list, fatal because tools render at position zero, so nothing caches across users.

The author's case was the first one in disguise: the timestamp came from a helper three call frames up the stack, written long ago for logging.

Where the breakpoint belongs

The rule is to mark the end of the shared portion of the prompt, never the end of the whole request. A top-level cache_control automatically slides to the last cacheable block, which suits a growing chat thread but fails for a large fixed preamble followed by a unique question: each request then writes a distinct entry over bytes nobody will ever read.

Further placement rules from the post: keep dates, modes, usernames and feature flags out of the prefix (on Opus 5 and Opus 4.8, dynamic text can instead be appended as a system message inside messages[], after the cached history); serialize tools deterministically and never add, remove or reorder them mid-conversation; remember caches are model-scoped, so switching models mid-loop forfeits the entire prefix; and note the minimum cacheable prefix is model-dependent — 512 tokens on the newest models, 1,024 on Opus 4.8 and Sonnet 5, and 4,096 on Opus 4.6 and Haiku 4.5 — with no error below the threshold, just silent non-caching. Requests allow at most four breakpoints.

Misses that survive a byte diff

Two mechanisms produce payloads that diff clean yet still miss, according to the post. Each breakpoint walks backward at most 20 positions looking for a prior entry; runs of consecutive tool_use or tool_result blocks each count as a single position, so heavy parallel tool calling is fine, but a sequential turn that appends more than 20 positions loses the previous entry entirely. Separately, in parallel fan-out an entry only becomes readable once the first response starts streaming.

Making it a standing check

The ground truth is usage.cache_read_input_tokens. The three fields partition the prompt — total tokens equal input_tokens plus cache_creation_input_tokens plus cache_read_input_tokens — so small input_tokens alone proves neither cheapness nor breakage. In a healthy multi-turn loop, reads grow turn over turn, writes bill only the delta past the highest hit, and plain input tokens are a small tail. If cache creation instead sits near the full conversation size every turn, something upstream is rewriting the prefix.

The operational advice is to keep an integration test that sends two byte-identical requests and asserts the second reports nonzero cache reads, and when hunting a break, diff the overlapping region of consecutive logged request bodies after stripping cache_control markers. The first divergence inside the overlap is the invalidator.

Why it matters

In tool-heavy agents, input tokens are most of the cost, and prompt caching is one of the few real levers — but this failure mode is silent. Requests succeed, latency is unchanged, and the bill simply drifts upward. The most expensive scenario the post describes is not a bad first implementation but a working one that regresses months later when someone adds a feature flag to the system prompt. Treating the usage object as a monitored signal, not a log artifact, is what separates a discount from a surcharge.