Anthropic Batch API postmortem: 112 of 1,842 jobs failed silently

What happened

A developer has published a postmortem on dev.to explaining how 112 of the 1,842 requests sent through Anthropic's Batch API over 30 days never produced a usable result — without a single visible error. The author builds and runs Preterview, a platform that conducts mock voice interviews and returns written reports, and had moved both the nightly portfolio scoring and the post-interview report generation onto the batch API.

Everything looked healthy on the surface: batches completed, the worker ran, the error logs stayed quiet. The first real signal was a user emailing a screenshot of an empty report with a blank score field, which traced back to a database row that had been sitting at NULL for four hours.

Where the 112 requests went

According to the dev.to post, the 30-day run covered 96 batches. Raw results: 1,747 requests returned a succeeded status, 71 errored, 24 expired at the API's 24-hour ceiling and none were canceled. Of the 71 errors, 52 were overloaded_error, 14 were invalid_request_error caused by transcripts exceeding the author's own token budget, and 5 were generic api_error.

The remaining 17 failures hid inside the successes. Those responses carried stop_reason: max_tokens, meaning the output was cut off mid-JSON. The developer's parser threw on the truncated bodies, an exception handler logged at DEBUG level, and the rows quietly stayed empty. The genuinely usable count was 1,730 — 93.9% — leaving a 6.1% failure rate that existing monitoring never saw.

The bug that crossed user data

The most damaging defect produced wrong results rather than missing ones. Batch results come back in arbitrary order, but the original code zipped the results list against the submitted sessions, assuming the two aligned. While every request succeeded, the ordering was usually close enough that nothing looked wrong. As soon as one request dropped out, every entry after it shifted up a slot, saving one candidate's report against another candidate's session.

Two sessions were affected before the author noticed, after reading a report that praised a Kubernetes project the candidate had never mentioned. Both users were contacted directly. The post describes this as the bug that actually caused fear: no exception, no alert, just plausible-looking output attached to the wrong person.

The fixes

The corrected code, shared in the post, follows a few rules:

• Match results by custom_id, never by position. custom_id should also be an opaque internal identifier — the author used a session UUID — not something convenient like an email address.
• Handle every value of result.type (succeeded, errored, expired, canceled) explicitly, with no default branch that lets a request slip through.
• Check stop_reason even on successes and treat max_tokens as a truncation failure to retry.
• Write a pending row to the database before submitting the batch, so a request that never returns shows up as a visible gap rather than an invisible hole.

The author's broader conclusion is that the reconciler — a table tracking custom_id, batch_id, status and attempt counts, swept by a scheduled job — should be the first component built for any batch pipeline, not an afterthought.

Latency and the retreat to synchronous

Batch latency over the run was heavily skewed: a median of 9 minutes, p90 at 51 minutes and p99 beyond six hours, with 24 requests sitting until the 24-hour processing ceiling and expiring. On that evidence, the post-interview report moved back to the synchronous API — someone who just finished a 25-minute interview will not keep refreshing for hours — while the nightly portfolio scoring stayed on batch, where the roughly 50% discount (the monthly bill fell from $209 to $104) justifies the wait.

Why it matters

Anthropic's Batch API offers asynchronous bulk inference at about half price, accepting up to 100,000 requests per batch within a 24-hour processing window. That is a strong deal for overnight work, but this postmortem shows that the contract differs from a synchronous API in ways that fail silently. A batch reaching its ended status says nothing about the fate of each request inside it, ordering is not guaranteed, and a successful response can still be truncated. Any team using the API — or any asynchronous queue with similar semantics — needs per-item reconciliation, exhaustive outcome handling and a mechanism to notice the requests that simply never came back.