GitHub trimmed Copilot agent costs by optimizing whole tasks, not tool calls

Optimizing the task, not the tool call

GitHub has published a detailed account of how it lowered the running costs of its Copilot coding agents without degrading results. Writing on the GitHub Blog, the team argues that token counts for individual interactions are a poor yardstick: a terse tool response can force an agent into extra calls and retries, making the overall job slower and more expensive. The right target, according to GitHub, is the completed task, from the user's request through to the final result.

The post describes four shipped changes built on that principle: reducing repetitive output while keeping useful context, stripping formatting that serves no purpose, shortening instructions without altering behavior, and delivering finished background work without a follow-up retrieval step. Candidate changes were first evaluated offline on agentic coding benchmarks and then validated through controlled online experiments. The examples come from GitHub Copilot CLI, but GitHub says the Copilot app and Copilot code review share the same underlying harness and picked up the same gains.

Why aggressive compression backfired

A cautionary experiment involved RTK (Rust Token Killer), a utility that shrinks shell output before an agent reads it. In GitHub's harness and benchmark setup, the shortened responses sometimes omitted details the model needed, prompting it to reopen the saved output or rerun the command. Those recovery steps added turns and carried more context forward, so although each individual response was smaller, tasks on average consumed more tokens and took longer. GitHub is careful to scope the finding to the integration and workloads it tested, not to every RTK configuration or to output compression generally.

A selective compressor with a safety net

Analysis of benchmark runs showed that install, build, test and lint output tends to be repetitive noise, while source-like output and arbitrary command results more often contain what the agent actually needs. That insight shaped a selective compressor. Early versions were too aggressive — a filter on git diff was pulled after agents kept reopening the original output — and the shipped version follows a three-part policy: commands such as cat, git diff, git show and arbitrary scripts pass through unchanged; search results from tools like grep are reorganized without dropping any match; and install, build, test and progress output is compressed only when the savings are substantial.

The agent can always retrieve the full original through a recovery path, which doubles as an evaluation signal: GitHub tracked whether agents opened saved originals, reran commands or took extra turns. Offline, tasks where compression triggered showed no statistically significant success regression and agents almost never opened the originals; online, average cost dipped slightly with no material regression in tracked quality metrics.

Deleting dead formatting

The cleanest win came from the view tool, which reads file contents into context. It prefixed every line with a number — a leftover from earlier editing tools that targeted changes by line number. Current editing tools match surrounding code instead, so the prefixes were removed. Offline benchmarks showed model-inference cost falling by roughly 5%, with success rates inside normal run-to-run variance. An online experiment with Copilot CLI users cut average daily model-inference cost per user by about 3%, with no material regression in quality or satisfaction metrics.

Slimmer prompts, same behavior

Prompts are resent on every turn, so GitHub also tackled accumulated instructions around the task tool, which spawns specialized agents for parallel work. By having Copilot iteratively rewrite its own prompt, that guidance shrank by roughly half, with the requirement that behaviors developers rely on remain intact.

Why it matters

For teams running AI coding agents at scale, per-call token counts are an attractive but misleading optimization target — GitHub's own data shows aggressive trimming can raise total cost. The post also demonstrates where the cheapest wins hide: formatting and instructions that outlived their purpose. Just as valuable is the methodology, combining offline benchmarks, controlled online experiments and recovery-path telemetry as a quality signal, which any team can reuse when evaluating changes to its own agent pipelines.