Litelm rebuilds LiteLLM routing in about 2,900 lines with two dependencies

A slim reimplementation of LiteLLM's core

A new open-source project called Litelm, which reached Hacker News's front page on 11 September, offers a radically trimmed alternative to LiteLLM, the widely used Python library for calling many LLM providers through a single interface. According to the project's GitHub repository, Litelm reproduces LiteLLM's routing and message-translation core in roughly 2,900 lines of code with just two required dependencies: the OpenAI SDK and httpx.

The README's argument is that LiteLLM's essential job is small — take a request such as openai/gpt-4o, route it to the correct endpoint, translate messages into the provider's native format, and return a normalized response — but that this logic sits inside more than 100,000 lines that also carry a proxy server, caching layers, cost tracking and a long list of features most users never touch. Litelm keeps the call path and discards the rest: there is no Router class, no proxy and no caching.

What is kept and what is cut

The retained surface covers model routing, message translation for Anthropic, Bedrock, Cloudflare and Mistral, streaming together with the stream_chunk_builder helper, tool calling, embeddings, text completions, the OpenAI Responses API and mock responses. Every function has an async variant — acompletion, aembedding, aresponses and atext_completion.

What disappears is the operational layer: the Router class with its load balancing and fallbacks, the proxy server, caching, budgeting and cost tracking, token counting, image generation, audio, OCR, fine-tuning, and the agents, guardrails and scheduler components.

Crucially, the API mirrors litellm's, with matching function names, arguments and response types. The README describes migrating as a one-line import change for existing litellm users.

Providers, setup and errors

The base install pulls in only openai and httpx; optional extras add the Anthropic SDK or boto3 for Bedrock. Calls use a provider/model-name string, and the project supports 19 providers, including OpenAI, Anthropic, Groq, Mistral, xAI, OpenRouter, Azure, Bedrock, Cloudflare, Together, Fireworks, DeepSeek, Perplexity, DeepInfra, Gemini, Cohere, Ollama, vLLM and LM Studio. Any OpenAI-compatible server can be targeted through an api_base parameter, which covers local inference stacks.

The README's provider table marks only seven of these as verified — OpenAI, Anthropic, Groq, Mistral, xAI, OpenRouter and Azure — with the remainder, including Bedrock and Gemini, listed as unverified.

Provider errors are mapped onto an exception hierarchy using the same names litellm uses, such as RateLimitError, ContextWindowExceededError and AuthenticationError, so existing error handling should carry over unchanged.

How compatibility is verified

The project labels itself alpha. The README reports 262 passing tests of its own, 75 passing tests ported from LiteLLM's suite against an upstream baseline, and in live runs 45 provider tests plus 10 DSPy smoke tests passing under the current dependency lock.

A maintainer attestation dated 11 September 2026 describes reviewing LiteLLM's routing and formatting changes across 360 core-path commits, closing compatibility gaps with tests written first. The attestation is explicitly scoped: it covers litelm's declared routing, formatting and DSPy surface, not full litellm compatibility. DSPy drop-in behaviour is claimed across seven execution paths.

The README is also unusually transparent about authorship. It says much of the code was written with Claude Code using Claude Opus 4.6/4.7, and that code from 14 May 2026 onward was written through Pi using GPT-5.5. It stresses that compatibility claims rest on tests and maintainer review, not on how the code was produced.

Why it matters

LiteLLM has become a common way for Python developers to avoid provider lock-in, but its breadth carries costs: a large install, a wide dependency tree and a codebase that is hard to read end to end. Litelm's existence suggests the part most applications actually use — routing and format translation — is a compact problem, and that a small implementation can stay compatible enough for a one-line swap. For teams that never deploy the proxy, the budgeting tools or the Router, that is an attractive audit and maintenance proposition.

The caveats are real. The project is alpha, most of its listed providers are unverified, and anyone relying on load balancing, fallbacks, cost tracking or the proxy server will not find those features. What Litelm does offer is a demonstration that the core abstraction is simple — plus an early example of AI-assisted development paired with explicit, test-based compatibility attestation.