BrighTO-Router: an open-source Rust LLM gateway for self-hosted AI traffic

A developer has released BrighTO-Router, a free, open-source gateway for large language model traffic written in Rust. In an announcement published on dev.to, the author says the goal is to give a team a single self-hosted API endpoint for all of its AI calls, while keeping the request path lean, quick and easy to reason about.

What the gateway provides

According to the dev.to post, BrighTO-Router exposes an OpenAI-compatible API surface covering chat, completions and embeddings, so existing OpenAI SDKs and clients can point at it without modification. It also routes traffic for Anthropic's Messages API, can forward calls to local or custom OpenAI-compatible backends, includes adapters for reranking services, and proxies automatic speech recognition and transcription requests.

The operational features are aimed at teams rather than individual users. The gateway supports team-level API keys and token budgets, records usage metadata in a PostgreSQL ledger, and groups models into what the project calls Model Groups, which can be balanced with round-robin or weighted distribution. The whole system installs as a single Docker container.

Why Rust

The author's argument for Rust is that a gateway sits between applications and models, so any overhead it introduces lands on every request. Rust brings predictable memory use and fast asynchronous networking, and its footprint is small enough that the entire router fits inside one Docker image.

Benchmarks isolate the router, not the model

The project publishes benchmark results generated against mock backends with deterministic behaviour, using payloads of up to one million tokens. Because the backends are simulated, the reported numbers reflect the overhead added by the router itself rather than model inference speed, which the author argues makes the measurements more reproducible. As with any self-reported benchmark from a project's own maintainer, the figures should be treated as indicative until independently verified.

Availability

The source code is hosted on GitHub under the BrighTO_Router name, alongside a separate documentation site and a prebuilt Docker image for deployment. The announcement comes directly from the project's author on dev.to, so claims about performance and readiness currently rest on the maintainer's own account.

Why it matters

Teams adopting LLMs usually end up juggling several providers and model families, each with its own keys, quotas and usage records. A gateway collapses that sprawl into one endpoint, and a self-hosted, open-source option appeals to organisations with data-handling constraints or those wary of adding yet another managed vendor to the stack. The OpenAI-compatible surface keeps switching costs low, since most tooling already speaks that dialect.

The Rust choice speaks to a real concern: a proxy that adds noticeable latency to every AI call undermines its own purpose, and a single-container deployment keeps the operational burden light. The trade-off is maturity. BrighTO-Router appears to be a young, single-maintainer effort announced in a blog post, so teams evaluating it should weigh community size, support and roadmap against the appeal of a fast, self-hosted control point for AI traffic.