BottleCap AI cuts Gemma-4-26B thinking tokens up to 69% at near-constant accuracy

BottleCap AI has released the fifth model in its ThinkingCap series, and the first built on a Google base: a retrained version of gemma-4-26B-A4B-it that, according to the company, needs 38.9% fewer thinking tokens on average — and up to 69.1% fewer — while giving up just 0.15 percentage points of accuracy. The post, which reached the Hacker News front page, is described by its authors as carrying the smallest accuracy trade-off of any ThinkingCap release so far.

Gemma-4-26B-A4B is Google's sparse mixture-of-experts model, with 26 billion parameters of which roughly 4 billion are active per token. BottleCap AI says its training objective was deliberately narrow: shorten the reasoning a model produces before it answers, and leave everything else — knowledge, answer quality, instruction following and safety behaviour — untouched. It is the third model family the company has applied the technique to, after Qwen and NVIDIA.

Results across twelve benchmarks

The company evaluated the original model and its own variant on the same harness and reports that every benchmark family produced shorter reasoning. Long-context and multimodal tasks saw the largest drop in thinking tokens at 54.4%, followed by knowledge and reasoning at 44.4%, maths at 36.1%, code at 27.6%, and instruction following with agentic work at 17.2%. The deepest single reductions were LongBench v2 at 69.1%, AA-LCR at 59.3% and MMLU-Pro at 49.1%; the shallowest were IFBench at 13.7% and tau-squared-bench at 20.8%. BottleCap AI attributes the smaller cuts to those two benchmarks involving work where extended reasoning matters least, such as satisfying format constraints or applying a fixed customer-service policy through tool calls.

Accuracy barely moved on average: 75.18% for the base model against 75.03% for the new variant. Eight of the twelve results fell within run-to-run spread, and five finished ahead of the original, including IFBench at +1.83pp and HMMT February 2026 at +1.28pp. The clearest regression is AA-LCR, down 4.00pp.

Looping also improved, according to the company: the base model repeated itself on 3.64% of responses versus 0.76% for the new variant, and on HMMT February 2026 the looping rate fell from 17.0% to 4.5%. Truncation rates dropped across the suite as well.

How it was measured

Because reasoning models are noisy to evaluate at high sampling temperatures, BottleCap AI ran multiple seeds per benchmark — from 4 on MMLU-Pro and MMMLU up to 64 on AIME 2026 and the HMMT sets — and reports accuracy as the seed mean with 95% confidence intervals. Both models ran on vLLM with identical settings: thinking enabled, temperature 1.0, top_p 0.95, top_k 64, and generation capped at 65,536 tokens. Notably, the base model's numbers were measured by the company itself rather than taken from Google's published figures, which keeps the comparison internally consistent but also means every result comes from the vendor, with no independent verification so far.

The company also plotted accuracy against a fixed thinking-token budget per question. It says its model's curve sits above the original's through the middle of the range — the same accuracy for a smaller budget — with the two converging once the budget is wide enough to constrain neither model.

A commercial tier

Alongside the open release, BottleCap AI advertises a PRO version that uses up to 87.6% fewer thinking tokens, 62% on average, positioned for faster answers and lower inference costs at scale and available through its enterprise contact. The blog post does not detail the PRO model's accuracy figures.

Why it matters

For reasoning models, chain-of-thought tokens are a large share of inference cost, and every token generated before the answer is latency a user waits through. A variant that trims roughly 39% of thinking while keeping accuracy within run-to-run noise translates directly into cheaper and faster serving, particularly on a sparse mixture-of-experts model that operators likely picked for efficiency in the first place. It can be dropped into an existing vLLM stack with no decoding changes. The broader signal is that token-efficiency tuning is becoming a product category in its own right, now spanning Qwen, NVIDIA and Google model families. The caveats are real, though: the numbers are self-reported, and the 4pp drop on AA-LCR means teams running long-context or agentic workloads should test before switching.