· via Hacker News – Front Page (native)
Cerebras pairs GPUs with its Wafer Scale Engines for five times more tokens per megawatt
Cerebras says disaggregated inference, now live on its cloud with AMD Instinct GPUs handling prefill and CS-3 systems handling decode, delivered five times the throughput per megawatt of a CS-3-only setup.

Cerebras splits inference between GPUs and its own chips
Cerebras has moved a disaggregated inference architecture into production on its inference cloud, dividing each large language model request across two kinds of hardware. According to a company blog post that surfaced on the Hacker News front page, GPUs handle prompt processing, the stage known as prefill, while Cerebras Wafer Scale Engine systems handle token generation, the stage known as decode. The company reports that the hybrid setup produced five times as many tokens per megawatt as a configuration running only its CS-3 systems, with no slowdown in generation speed.
Why separate prefill from decode
Every inference call runs in two stages. Prefill reads the user's prompt and builds an intermediate representation of the model's state; decode then generates the answer one token at a time, feeding each new token back in to produce the next. Because decode revisits the model's weights for every single token, the speed at which those weights reach compute becomes the limiting factor. Cerebras argues this favors chips built around on-chip SRAM, where weights sit beside the compute units, over GPUs that stream weights in from external HBM memory.
The company's headline specifications for its wafer-scale chips are tens of gigabytes of on-chip SRAM and tens of petabytes per second of memory bandwidth, which it says works out to 350 times the SRAM capacity and 5,400 times the memory bandwidth of a typical GB300-class GPU. Prefill, meanwhile, is a compute-heavy batch problem that conventional accelerators already handle well, which is what makes the division of labor attractive.
What the production numbers show
The measured deployment ran on the Cerebras Inference Cloud, with AMD Instinct GPUs processing prompts and CS-3 systems generating responses. According to the post, throughput per megawatt on a shared GPT-OSS-120B endpoint was tracked on November 14 before and after disaggregation was switched on, normalized against a single-system baseline; enabling the split lifted throughput to five times that baseline. Cerebras says it intends to roll the architecture out across hundreds of megawatts of capacity, in its own cloud and in partner datacenters, by the end of 2027.
The company frames the move as a response to a capacity squeeze, citing Goldman Sachs Research estimates that AI inference demand will grow roughly 24-fold by 2030 while datacenter capacity expands only about 2.2 times. In a power-constrained market, tokens per megawatt becomes the metric that determines how much demand a provider can actually serve. Cerebras points to interactive agentic applications such as Copilot and Codex as the workloads driving that demand.
A partner program, not just a cloud feature
Cerebras is also offering the architecture to infrastructure operators. Partners can co-locate their choice of prefill accelerator with Cerebras systems inside a unified cluster, following a Cerebras reference design, and integrate in one of two ways. An end-to-end software stack includes a routing framework that manages prefill and decode pools, a prefill engine built on open-source projects such as vLLM or SGLang, a Cerebras decode engine, and software called KV Portal that moves key-value caches between the two pools over RDMA. Partners who want more control can instead use a decode-only SDK, bringing their own inference framework and prefill engine, with a mock Cerebras replica and a reference integration supplied as development aids. Early-access partners named in the post include Amazon Web Services, Fireworks AI, Gimlet Labs and General Compute. The announcement sits alongside the recently introduced CS-4 system, expanded manufacturing, and a planned 700 megawatts of datacenter capacity.
Why it matters
Power, rather than chip supply, increasingly caps how much inference the industry can deliver, and a claimed fivefold gain in tokens per megawatt is significant if it holds outside Cerebras' own environment. The design also repositions Cerebras as a complement to GPU fleets rather than a replacement for them: operators keep their GPUs for prefill and add wafer-scale systems only where they pay off, in decode. That lowers the adoption barrier for hardware that has so far been a niche purchase. Latency-sensitive agentic applications benefit directly, since perceived responsiveness depends on decode speed. The usual caveat applies: the five-times figure is a vendor measurement on the vendor's own cloud, and independent benchmarks have yet to appear.
- #cerebras
- #inference
- #ai-hardware
- #datacenters
- #llm