One call at 99% still means a 100-call LLM job fails 63% of the time

A script that dies on page 71

A post on dev.to, written by altrouter, opens with a scenario anyone who has scripted against a model API will recognize: a program that summarizes a 100-page PDF, one call per page, times out on page 71. Rerunning the script means paying again for the 70 pages already processed, and the second attempt carries the same risk of dying somewhere as the first. The author's argument is that nothing in the system is actually broken — the provider simply had a bad moment — and that the failure is a predictable property of chaining many calls together.

Success rates multiply

According to the post, a job of N calls completes only if every call completes, which puts the job-level success rate at p to the power N, where p is the per-call success rate. The author is explicit that the per-call figures are assumptions, not measurements: 99% represents a call that fails only from network or server trouble, while 95% approximates a call whose response must also parse as valid JSON.

Even at optimistic rates, the compounding is harsh. Two calls at 99% finish about 98% of the time, and five calls still finish roughly 95% of the time. At a hundred calls, 99% per call drops the whole job to 36.6% — meaning it fails in about 63% of runs, the figure in the post's headline. At 95% per call, a hundred-call job completes just 0.6% of the time.

The price of starting over

To make the stakes concrete, the post prices a page at 1,000 input tokens and 250 output tokens. Using GPT-5.6 list prices — $2 per million input tokens and $12 per million output, read from an LLM gateway catalog on 29 October 2026, per the author — that works out to $0.005 per page, or $0.50 for the whole document.

Restarting from call one after each failure multiplies that cost badly. In the post's worked example, at 95% per call the 100-page job needs about 169 attempts on average, each dying roughly 19 pages in, for 3,358 paid calls: $16.79 spent on a $0.50 job. Retrying only the call that failed changes the expected cost to N divided by p — 105 calls, about $0.53 — nearly independent of job length. At 99% per call the contrast is milder ($0.87 versus $0.51 for a hundred calls) but has the same shape.

Checkpoint each answer, retry the call

The recommended fix pairs per-call retries with a checkpoint on disk. Retry the individual failed call a few times with a growing pause, and append every completed answer to a JSONL file, flushed to storage before the next paid request. On the next run, the script reads the file, skips pages already finished, and resumes where the previous run stopped, so a crash costs at most the single call in progress. The post includes a short Python function implementing this.

The author also warns against a tempting shortcut: catching the error and moving on. The job would always finish and never pay twice, but at 95% per call roughly five pages in a hundred would be quietly missing from the final summary.

Where the approach ends

The post lists its own limits. A page too long for the model will fail on every attempt, so the job should stop; the checkpoint preserves everything before it. The arithmetic assumes failures are independent, but outages arrive in bursts, which is why the sample code backs off exponentially and eventually gives up. Retries are only safe for idempotent steps — a call that also emails a customer should not be repeated blindly. And no gateway can do this checkpointing on your behalf, including the one the author works on (a disclosed interest), because a gateway sees individual calls, not your job.

Why it matters

As LLM usage shifts from single-shot demos to batch pipelines — summarizing documents, extracting structured data, classifying thousands of rows — the reliability habits of batch computing become part of everyday scripting. The dev.to post makes the arithmetic visible: a per-call success rate that feels fine in isolation collapses quickly at scale, and the difference between restarting a job and resuming it is not stylistic but an order of magnitude in cost. The remedy is small — a retry loop with backoff and an append-only results file — and it transfers to any long sequence of unreliable network calls, LLM or otherwise.