OpenRouter analysis finds same model priced up to 14x higher across providers

A pricing analysis published on dev.to has quantified how much the provider behind an API endpoint matters: on OpenRouter, a routing platform where a single model name maps to many providers, the identical model can cost up to 14.47 times more depending on which endpoint serves the call.

How big the spread is

The authors pulled per-provider pricing for every paid model on OpenRouter. Of the 405 paid models listed, 182 are served by two or more paying providers. Across that group, the median gap between the cheapest and most expensive endpoint is 1.87x, the widest is 14.47x, and 46% of the models vary by at least 2x. The weights, the model card and the API call are identical; only the invoice changes.

Why one model has many prices

According to the analysis, three variables move independently under a single model identifier:

• Precision. A provider serving an fp4 quantization is not delivering the same artifact as one running bf16, even though the model id matches. Lower precision is cheaper for the provider and changes what the caller actually receives.
• Uptime. An endpoint at 89.3% availability and one at 99.9% can both appear as up on a status page, yet they represent materially different reliability in practice.
• Data location. 71 of the 182 multi-provider models, or 39%, are served from providers based in more than one country. For anyone with data-residency obligations, the model identifier says nothing about where the data goes; the provider list does.

The post's argument is that none of this is concealed. It is scattered across separate pages that nobody typically assembles before committing to a model.

Latency has to be measured, not inherited

OpenRouter's public model data leaves the latency field null, which the authors read as an honest admission that the answer depends on provider, region and moment. So they called 329 models through a paid API with one fixed prompt and recorded time-to-first-token and tokens per second.

Two implementation traps are flagged along the way. With reasoning enabled, a model can spend its entire token budget on thinking and return empty content, so the benchmark yields nothing while the call still costs money. And undisclosed prices arrive as -1, which silently deflates any projected-cost sum unless the value is clamped at zero before adding. The team's single-call measurements are published next to OpenRouter's own p50 to p99 percentiles, but the two datasets are kept separate because they answer different questions: steady-state traffic versus one controlled request.

Fluent output that is wrong

The same project graded 330 models on seven axes of Korean-language quality, including register, honorifics, terminology and knowledge of Korean institutions. Two axes fail almost everywhere: only 8.5% of models handle honorifics correctly and 9.4% demonstrate adequate knowledge of Korean institutions. Terminology passes at 31.2%, format compliance at 47.6% and register at 53.3%.

The authors compare this failure mode to a bug that fails silently: output can read as fluent, natural Korean while getting the honorific wrong, and text that reads well but is incorrect will sail through reviews that visibly broken output would not. Nor do the usual proxies predict the result. According to the grading, gpt-3.5-turbo-16k, a 2023 release, scores a perfect 3.00, above most 2026 flagship models.

The data is open

Everything is available without an API key through the project's Hugging Face Space, ginigen-ai/open-router-leaderboard, covering 425 models in total: per-endpoint price, precision, uptime and provider headquarters, latency and throughput percentiles, plus the team's own measurements and per-axis grades. The dataset is refreshed daily, spans three languages and three currencies, and a full methodology writeup is posted on the Hugging Face blog.

Why it matters

Picking a model and picking an endpoint are two separate decisions, and the second one can move a bill by close to an order of magnitude on identical weights, shift reliability by several points of uptime and change a compliance posture across a national border. For teams building on routing platforms, quantization means "the same model" may not be the same artifact at all, so cost comparisons have to happen at the endpoint level, not the model level. The Korean results carry a broader lesson for anyone shipping LLM features: evaluation suites that check only whether a model responded, and not whether it responded correctly, will keep passing systems that fail quietly.