Gimlet Labs raises $300M Series B led by Andreessen Horowitz to scale AI inference

Gimlet Labs has raised a $300 million Series B led by Andreessen Horowitz, with a long syndicate alongside it including Sapphire Ventures, Menlo Ventures, 645 Ventures, Arm, Eclipse, Emergence, Factory, Hudson River Trading, M12, OnePrime Capital, Prosperity7, QuantumLight, Samsung Ventures, Tiger Global Management, Triatomic, Wing Ventures and XTX Markets. The company announced the round in a blog post that reached the Hacker News front page.

The raise comes just over five months after Gimlet's Series A. According to the company, since March it has added billions of dollars in contracted revenue, built a datacenter pipeline measured in gigawatts, and is scaling quickly toward hundreds of megawatts of managed capacity.

The problem Gimlet is attacking

According to Gimlet, monthly token generation has grown six-fold over the past twelve months, with some projections calling for another twenty-fold increase by 2030. Underwriting that growth is an industry investment program the company describes as already close to $1 trillion per year, with a cumulative $7 trillion expected by 2030.

Power is the binding constraint. Gimlet cites roughly 18 GW of AI datacenter consumption in 2025, a figure expected to triple by 2030, which in turn requires new generation, grid capacity and behind-the-meter energy production. Because those resources will eventually run into delivery limits, the company argues that maximizing throughput per kilowatt is how the industry keeps scaling.

There is also a latency dimension. Agentic workloads run multi-step inference loops with dozens of sequential model calls, so latency compounds across them. Model sizes have grown from a few hundred billion parameters a few years ago to 3-10 trillion today, and context windows have expanded from around 100K tokens in 2023 to a million or more.

Gimlet's core diagnosis is that inference has largely been served on homogeneous infrastructure repurposed from training clusters, which forces a choice between high throughput and low interactivity. The company argues inference is fundamentally different from training, and that infrastructure needs to be rebuilt from the ground up with inference in mind.

A multisilicon inference cloud

Gimlet describes itself as building the first "multisilicon cloud," designed from scratch for inference performance. The platform runs on heterogeneous hardware — GPUs, near-memory compute, dataflow architectures and CPUs — and its software stack decomposes models and schedules each phase of the inference workload onto whichever silicon architecture suits it best.

The company claims that breaking models apart this way delivers 5-10X speedups for the same power footprint, or similar throughput improvements at the same latency, and 3-10X faster performance for frontier workloads compared with traditional homogeneous infrastructure.

The post outlines several disaggregation schemes with different tradeoffs. Prefill/decode splits the compute-bound prefill phase from the memory-bandwidth-bound decode phase, and offers the lowest latency. Speculative-decode disaggregation and attention-FFN disaggregation provide speedups over homogeneous deployments with higher throughput. Gimlet says models are traced and decomposed, then scheduled based on workload SLAs and available hardware, with dynamic rebalancing so compute is not stranded: if one hardware type is fully utilized for decode and demand grows, decode instances can spin up on other available hardware. The company says it is working on new disaggregation types it will detail in future technical posts.

Gimlet says it currently works with frontier labs and other large-scale consumers of inference, and is ramping capacity to reach a broader audience. It is also hiring across systems, datacenters, compilers, distributed systems, networking and performance engineering.

Why it matters

This is one of the larger venture bets placed specifically on inference infrastructure rather than frontier models or training compute. If token demand grows the way Gimlet projects and power remains the bottleneck, efficiency per watt becomes the decisive competitive axis — and Gimlet's claim that heterogeneous silicon plus granular disaggregation can unlock order-of-magnitude gains is a direct challenge to incumbent GPU clouds built on repurposed training infrastructure.

The reported traction — billions in contracted revenue within months of a Series A — suggests frontier labs are already shopping for inference alternatives. The open questions are whether disaggregated multisilicon scheduling can be delivered reliably at a scale of hundreds of megawatts, and whether the performance claims hold up as homogeneous stacks keep improving. The $300 million from Andreessen Horowitz and its co-investors is a wager that they will.