Hetzner launches free experimental OpenAI-compatible inference API for open-source LLMs

What Hetzner launched

German hosting provider Hetzner has opened a free, experimental endpoint for running open-source large language models. According to a dev.to walkthrough, the service was announced on July 24, 2026 as part of Hetzner's new "experiments" platform, and runs on the company's own infrastructure in Germany and Finland. It costs nothing for as long as it keeps its experimental status, and Hetzner has said it will notify users by email in advance with details of any billing changes.

The integration model is deliberately lightweight. A developer creates a token at Hetzner's experiments portal, points any OpenAI-compatible SDK at the base URL, and sends ordinary chat completion requests. Hetzner handles serving, batching and scaling behind the scenes, so there is no GPU to rent, no model weights to download and no inference server to configure.

The model on offer

As of mid-August 2026, the endpoint serves exactly one model: Qwen/Qwen3.6-35B-A3B-FP8, built by Alibaba's Qwen team and licensed under Apache 2.0. Hetzner's documentation, as summarized by dev.to, describes it as a Mixture-of-Experts model with 35 billion total parameters, of which roughly 3 billion activate per token. It offers a 262,144-token context window, FP8 quantization to reduce memory use, and multimodal support that accepts images alongside text in the same request.

Hetzner says the single-model lineup is intentional: it wants to validate the platform, gather feedback and confirm reliability before adding further models based on demand and operational experience.

Rate limits and measured speed

The service enforces per-key limits and returns HTTP 429 when they are exceeded. Every 60 seconds, each key allows 3 million input tokens and 60,000 output tokens; over 24 hours, the allowances rise to 500 million input and 5 million output tokens. The dev.to author calculates that the daily input quota alone would cover roughly 250,000 requests at 2,000 input tokens each, while the per-minute output cap is the more likely bottleneck for sustained generation.

Independent testing by Sliplane.io on July 23, 2026 reported a 153 ms median time-to-first-token across seven short requests, and 224 output tokens per second across five longer generations capped at 512 tokens. The tester cautioned that this was a single-client snapshot rather than any form of guarantee.

Explicitly not production-grade

Hetzner's terms for the experiment, as quoted by dev.to, describe the service as offered "as is" with no guaranteed performance or availability, particularly during periods of high demand. No backups are created, the service can be changed or withdrawn at any time without notice, and the company explicitly tells users not to run production environments on it. Future token prices have not been published.

The dev.to write-up also flags that Hetzner has not publicly documented a Data Processing Agreement for the experimental endpoint. That matters for anyone weighing whether GDPR-regulated personal data can legally be sent to the service, and it rules out the endpoint for most customer-facing work regardless of the missing SLA.

Why it matters

A zero-cost, OpenAI-compatible endpoint from an established EU host lowers the barrier to experimenting with open-weight models to almost nothing. Existing code written against the OpenAI SDK works after changing a single base URL, and the daily input quota is large enough for serious prototyping without a credit card or a GPU bill.

It also signals where Hetzner may be heading. The company built its reputation on aggressively priced dedicated servers, and an inference experiment suggests it is testing whether it can compete in AI serving the same way. If the service graduates into a paid product, the pricing pressure on established inference providers could be considerable.

The caveats are just as real. One model, no SLA, possible withdrawal without notice and no documented DPA mean this is a sandbox, not infrastructure. Developers who need EU-hosted inference for production will still require a commercial provider, but for evaluation, side projects and learning, it is a remarkably cheap way in.