Cost of a fixed level of AI performance falls 47% per quarter, Epoch AI finds

A fixed level of capability keeps getting cheaper

Epoch AI researchers Emberson and Roodman estimate that the cost of reaching a given level of AI performance has fallen by an average of about 47 percent per quarter over the past three years. Compounded, that works out to roughly a 13-fold drop each year, a slope Marginal Revolution, summarising the report, describes as steeper than anything earlier transformative technologies managed.

The report's flagship example: OpenAI's o3 cost roughly $0.30 per question to score 75 percent on the GPQA Diamond benchmark in January 2025, while GPT-5.6 Luna reached roughly the same score for about $0.0004 per question in mid-2026. That is a decline of around 725 times in under 18 months.

Marginal Revolution draws a second conclusion from the same data: the apparent threat from open-weight models looks smaller once you account for this curve, because frontier models are not just smarter than older ones, they are also becoming far cheaper to run at any fixed quality level.

Where the gains come from

A separate technical analysis published at jyn.dev set out to test whether the deflation is real, and traces it to several compounding layers rather than a single cause.

Hardware contributes a familiar curve. GPU power efficiency has been doubling roughly every two years, a pace the analysis compares to Moore's law in the 1960s.

Model economics are more nuanced. Per-token prices for frontier models are not consistently falling, but the cost of completing a whole task is, because newer models need fewer tokens, retries and corrections for the same job. Pareto-frontier charts in the analysis show that between 2025 and 2026 the quality axis stayed roughly flat while the cost axis moved about two orders of magnitude cheaper.

Inference software is an underrated factor. Engines such as vLLM improved energy efficiency by about 40 percent in the 15 months between versions 0.5.4 and 0.11.1, NVIDIA reported up to 50 percent efficiency improvements between MLPerf submissions 2.0 and 2.1, and Intel demonstrated a 2.4x throughput increase between MLPerf 6.0 and 6.1 on unchanged hardware. Across engines, the analysis puts yearly gains at 10 to 50 percent.

Architecture changes compound the rest. Mixture-of-experts designs can match a dense model's benchmark performance with about seven times fewer active parameters (a 6B model behaving like a 0.8B one at run time), though they help hosted deployments more than local ones, because all experts still have to sit in memory. Mamba and Mamba-transformer hybrid models compress context into a lossy running summary instead of remembering every token, cutting memory needs by five times or more; the analysis cites Nemotron-H-47B holding over a million tokens in 32 GB of VRAM.

What it means for software

The jyn.dev analysis makes three projections: large language models get woven into computing as infrastructure rather than sold as a product within the next year or two; models at today's frontier quality run locally on commodity hardware within three to six years; and sheer token volume stops being the bottleneck, with model quality and access becoming the constraints instead.

For engineering teams the practical consequence is that cost assumptions baked into architecture decisions now have a shelf life of months. Features once gated behind the question of whether they justify a model call become cheap enough to enable by default, and elaborate batching or caching workarounds lose much of their justification. The analysis adds a caveat: users tend to spend the savings on better output rather than lower bills, so raw token consumption per unit of energy stays roughly flat even as quality per unit of energy climbs.

Why it matters

A roughly 13-fold annual drop in the price of a fixed capability changes the planning question from whether a team can afford inference to which quality tier and access path it should build around. It also weakens the pure price advantage of open-weight models, since frontier inference prices are collapsing on their own schedule. One distinction matters when applying the headline number: per-token prices at the frontier are not reliably falling, so products that depend on cutting-edge models will see relief through cheaper tasks and cheaper capability levels rather than uniformly cheaper tokens.