Re-sent tool definitions can make up 32% of an AI agent's API bill

Tool schemas ride along on every request

OpenAI-compatible chat APIs are stateless: the model retains nothing between calls. Every request must therefore carry the system prompt, the conversation history and the complete array of tool definitions, each written as a JSON Schema object with a name, a description and typed parameters. An agent loop issues one request per step — the model picks a tool, the host runs it, the result is appended — so a ten-step task ships ten identical copies of the toolbox.

A dev.to post by altrouter, an API reseller, quantifies what that repetition costs. The author assembled 18 tools typical of a coding agent — file reads, writes and edits, directory listing, code search, shell and test execution, three git commands, web search and fetch, an HTTP client, a database query, issue creation, messaging and a user-prompt tool — with ordinary one- and two-sentence descriptions. Tokenizing them with tiktoken's o200k_base encoding, the tokenizer behind current GPT models, produced 2,388 tokens per request: roughly 133 per tool on average, ranging from 54 tokens for a parameter-less git_status to 193 for a five-parameter search_code. That payload is transmitted before the user's task has even been read.

The arithmetic on a ten-step run

The post then models a modest agent run: a 500-token system prompt, a 200-token task, and ten steps that each add about 780 tokens of history and produce 150 output tokens, priced at the GPT-5.6 rates the author cites for 15 October 2026 — $2.00 per million input tokens and $12.00 per million output tokens.

Across ten steps the schemas total 23,880 tokens, 36% of all input for the run. In dollars they account for about $0.048 of a $0.150 run, or 32% of the bill. Per request, the toolbox outweighs the system prompt and task combined by more than three to one. Scaled to 1,000 runs a day, that is roughly 716 million tokens a month spent re-sending the same function descriptions.

Three fixes, and where they stop working

The first fix is pruning. A code-fixing task has no need for issue creation, messaging or database access, and cutting the toolbox from 18 tools to the 6 the task actually uses dropped monthly schema spending from $1,433 to $563 — $870 saved — while the per-run cost fell from $0.150 to $0.121, against a $0.102 baseline with no tools attached.

The second is trimming descriptions. Keeping only the first sentence of each and removing per-parameter text shrank the payload from 2,388 to 1,545 tokens, a 35% reduction. The author cautions that descriptions are how the model decides which tool to call, so selection accuracy should be re-tested after the cut.

The third is simply measuring: the post includes a short tiktoken script that turns any tools array into a per-tool token count in about a minute.

The author is candid about the limits. Providers do not forward the JSON verbatim — they rewrite it into internal formats and append their own instructions — so true counts differ by vendor and should be checked against usage.prompt_tokens on real calls made with and without tools. Prompt caching changes the money considerably: the tool list sits at the front of every request, exactly the region that caching discounts, so the 32% share shrinks when caching applies. But swapping toolboxes between steps invalidates that cache, which means pruning tools and maximizing cache hits pull against each other. It is also worth noting the post's publisher sells discounted GPT-5.6 access, giving the analysis a commercial angle, though the measurement method itself is reproducible.

Why it matters

Agent frameworks tend to treat the tools array as fixed configuration, but this analysis shows it behaves like a recurring cost that scales with every step of every run. The 32% figure comes from a single self-published benchmark and will vary with toolbox size, step count, pricing and caching, yet the levers are unusually cheap: pruning and tightening schemas require no model change and no prompt rewrite. As multi-step agents become the default interface for coding and operational work, input overhead may decide whether a given agent design is economically viable at scale. The caching tension the post identifies is also a genuine design constraint — teams optimizing tool lists and teams optimizing cache hit rates need to coordinate, because each can quietly undo the other's savings.