Experiential open-sources an OpenAI-compatible gateway that learns to route agent traffic

What Experiential does

Experiential Labs has released its model gateway and router as open source. The project, announced via a Show HN post pointing to its GitHub repository, is built for agent workloads: it sits as a single layer in front of every model a team might use — hosted providers, models reached with your own keys, local models, and custom models.

According to the README, the gateway exposes all of them through one OpenAI-compatible API, lets administrators decide which users and agents may call which models for which use cases and under what spending limits, and — the headline feature — can use production traffic to fit a custom router, or a fine-tuned model, that trades off quality, speed, and cost for a specific workload. The Hacker News title frames the pitch bluntly: a post about turning usage into a better model.

Local gateway in a few commands

The local path is deliberately short. Running pip install experiential followed by exp starts a gateway on loopback, with a first-run wizard that handles provider, model, and reasoning-effort selection, remembers each provider connection, and proposes defaults for a public alias, an identity, and a $50.00 command budget before printing a one-time key. Requests then look like any OpenAI call — the README demonstrates a curl against a local /v1/chat/completions endpoint using a model alias such as opus-5.

Python users can skip raw HTTP entirely: exp.load_router("my-project") returns a client that behaves like the official OpenAI SDK, backed by a fitted project router served from the gateway's compiled native data plane.

Hosted option and agent-driven setup

Teams that would rather not operate the gateway themselves can use the hosted platform, which the README says serves the same OpenAI-compatible API — plus Anthropic's Messages format — from the company's managed endpoint. The setup documentation ships copy-paste prompts meant to be handed to a coding agent such as Claude Code, Cursor, or Codex, so the agent performs its own configuration.

Bring-your-own-key covers OpenAI, Anthropic, Gemini, Azure, Bedrock, Fireworks, and OpenRouter, with traffic passing through at no added charge, per the README. The onboarding flow also lets teams upload existing LLM traces to the platform as telemetry, import their historical spend, and then repoint tools including Aider and Conductor — along with the coding agents above — at the gateway.

The optimization loop

What separates Experiential from a generic aggregator is the feedback loop. The starting point is OpenTelemetry traces from your current agent; for anyone without traces to hand, Experiential has published a public terminal-tasks OTLP dataset on Hugging Face. A command like exp build support-agent then constructs a simulation from those traces and optimizes a router against it, walking through providers, models, and budget along the way.

The loop continues from there: once the router is serving real traffic and generating its own traces, exp optimize model support-agent fine-tunes an open source model you own using Tinker. The gateway's logs become training signal, and the end state is a model shaped by your actual request distribution rather than a generic one.

Telemetry and project hygiene

Anonymous, aggregate PostHog product telemetry is enabled by default. The README states it never includes prompts, traces, actions, observations, paths, model names, credentials, or raw customer content, and it can be disabled with exp config telemetry disable, with the preference stored locally in .exp/settings.toml. The repository is developed with uv, formatted and linted with ruff, type-checked with ty, and tested with pytest.

Why it matters

Routing has become one of the busiest corners of LLM infrastructure, and most gateways compete on catalog breadth or price. Experiential's bet is different: that the routing layer should learn from the traffic it carries, with your own traces producing both the routing policy and, eventually, a fine-tuned model. If fitted routers measurably outperform static rules on real workloads, that is a genuine moat — nobody else has your logs. The OpenAI- and Anthropic-compatible surfaces keep switching costs low, and the agent-directed onboarding makes the target audience explicit: teams whose coding agents are burning budget across a dozen models and need one place to observe and shape that spend. The open question, as with any learned system, is evaluation — whether a simulation built from yesterday's traces keeps up with tomorrow's traffic.