Kinetic-4B beats Claude Haiku 4.5 and GPT-OSS-120B on tool-calling benchmark

A 4B model wins a narrow race

A four-billion-parameter open model called Kinetic-4B has outscored Anthropic's Claude Haiku 4.5 and OpenAI's GPT-OSS-120B on a tool-calling benchmark while responding far faster than either. According to a dev.to writeup summarising an evaluation published by Bengaluru lab Conscious Engines on 1 April 2026, Kinetic-4B reached 82.33% accuracy on a 300-sample test with a p95 latency of 1.61 seconds. Claude Haiku 4.5 scored 80.0% at 4.02 seconds, and GPT-OSS-120B, with roughly 30 times the parameters, came in at 76.33% and 7.99 seconds.

The headline accuracy gap is small. The failure rate is not: 4.67% of Kinetic-4B's calls failed versus 9.67% for Haiku 4.5, per the same writeup. In agent pipelines a failed call is a malformed or unusable request that has to be caught and retried, and each retry compounds the delay across a chain of steps. Halving that rate changes the amount of error handling an agent builder has to write.

How the evaluation was built

The test set drew 300 samples from Composio, an agent tool-integration platform covering more than 800 integrations including GitHub, Slack, Salesforce, Stripe, Notion and Jira. Each prompt offered ten candidate tools — one correct answer plus nine distractors taken from the same toolkit, mirroring the real-world failure of picking a near-identical tool. All models saw the same samples, seed and tool_choice=auto setting.

One row in the results table tempers the story. Qwen3-4B-Instruct-2507, the unmodified base model Kinetic-4B was fine-tuned from, already scored 78.67% on its own. Of the 300 samples, 222 were answered correctly by both the base and fine-tuned models; the fine-tune fixed 25 and broke 14, a net gain of eleven. In other words, most of the capability was already sitting in the open base model.

One rented GPU, four and a half hours

The training run behind the result was modest. Conscious Engines used LoRA at rank 64 and alpha 128 across all projections, training 132 million parameters — about 3.18% of the 4.15-billion total — at a 2e-4 learning rate for two epochs. The run took roughly 4.5 hours on a single rented GPU, using a dataset of 13,694 samples built from Composio's top-20 toolkits with the same one-correct-plus-nine-distractors structure.

An earlier attempt at a full fine-tune on generic function-calling data managed only about 71%, below even the untouched base model. Freezing the base weights and adapting through a low-rank update preserved instruction-following that the full fine-tune had eroded — the transferable lesson being that LoRA is the safer default for narrow tasks on top of a strong base.

The adapter, published as consciousengines/Kinetic-FC-LoRA on Hugging Face, applies on top of Qwen3-4B-Instruct-2507, so the model can be reproduced rather than taken on trust.

Who is behind it

Conscious Engines works out of Indiranagar in Bengaluru and was founded by Kautuk Kundan, previously of Ethereum-scaling company Stackr Labs. The lab is backed by a16z, and the founder's public statements put annualised revenue near $1 million. The Kinetic-4B work came out of a LossFunk residency, authored with Ritam Pal.

The lab's commercial thesis is models matched to each use case, with frontier models serving as teachers for distillation rather than as the thing you ship. On cost, the dev.to writeup notes Haiku 4.5 lists at $1 per million input and $5 per million output tokens, which is cheap in absolute terms; Conscious Engines counters that large enterprise discounts in the 75-80% range obscure the true unit cost and encourage lock-in. The founder has also described spending around $35 in API tokens on a task a local model ran at no marginal cost, though that is a self-reported anecdote rather than a measured comparison.

Where the result does not apply

Three caveats are worth stating plainly. The evaluation is a single 300-sample set, self-published by a party with an interest in the outcome, and not independently replicated. It measures one capability — Anthropic reports 73.3% for Haiku 4.5 on SWE-bench Verified, and a tool router is not a coding model. And the nine-distractor design matches the setup Kinetic-4B was trained on, while production traffic is messier.

Why it matters

The practical takeaway for agent builders is routing: send structured, repetitive tool calls to a small fine-tuned model, keep a general model behind it for anything ambiguous, and measure your own p95 rather than trusting anyone else's table. The result also shows how cheap narrow capability has become — a single rented GPU for an afternoon — and how much of that capability already exists in open base models, waiting to be unlocked.