Cerebras explains how it serves 3-trillion-parameter models across 59 wafer-scale chips

Trillion parameters, 44 GB at a time

Cerebras has published an engineering breakdown of how it deploys trillion-parameter models on wafer-scale chips that carry only 44 GB of on-chip SRAM each. The post, which reached the Hacker News front page, uses the open-source Kimi K3 model — roughly 3 trillion parameters — as a worked example, and states that multi-trillion-parameter models run at 1,000–2,000+ tokens per second on the company's hardware.

The memory arithmetic

A single WSE-3T wafer holds 44 GB of on-chip memory, while trillion-parameter frontier models need terabytes for their weights alone. Cerebras closes that gap by splitting the model across many wafers.

Kimi K3 has 69 linear-attention layers and 24 dense layers — 93 in total. Assuming a 256k-token average request size, 4-bit weight quantization (as in the Hugging Face implementation) and an 8-bit KV cache, 3 trillion parameters at 4 bits each come to about 1.5 TB of raw weights. Divided across 93 layers, that works out to roughly 16 GB per layer.

Layer type then decides placement. Dense, full-attention layers must store keys and values for every token of every active request, so Cerebras places one dense layer per wafer and reserves the remaining SRAM as KV cache. Linear-attention layers hold only a small, fixed-size state per request regardless of context length, so two of them — 32 GB — fit on a single wafer. The 69 linear layers therefore occupy 35 wafers, the 24 dense layers take 24 more, and the whole model fits on 59 wafers.

Weights stay put, activations move

The wafers are chained into a pipeline. The Cerebras compiler maps model layers onto regions of wafers, and the runtime charts the path tokens take between them. Because a 16 GB layer fits inside a 44 GB wafer, weights typically never move and layers are not split across chips — only activations travel, over RDMA on Ethernet at roughly two microseconds per hop.

The company likens the setup to an assembly line: wafer one computes layers 1–2 and hands its activations to wafer two, which handles layers 3–4, and so on until the last wafer emits the next token.

Filling the pipeline

A natural objection is that a sequential pipeline would leave most wafers idle while waiting their turn — effectively a batch size of one. Cerebras instead keeps the pipeline full of different requests: while stage three computes a token for request A, stage two works on request B and stage one on request C. Production systems use many more, finer-grained stages than the one-per-layer illustration, and each stage processes a micro-batch of requests rather than a single one. Requests in flight are roughly the number of stages multiplied by the micro-batch size; smaller micro-batches push per-request latency down, while larger ones raise aggregate throughput.

Concurrency is ultimately bounded by memory. Every in-flight request needs its own KV cache on each dense-layer wafer, so the spare capacity — 44 GB minus the 16 GB layer, about 28 GB per dense wafer — sets the ceiling. Cerebras divides that room by the cache footprint of a 256k-token request to get the supported concurrency, though the published post does not spell out the final figure.

The company also cautions against reading too much into concurrency or batch size on their own. According to Cerebras, the more meaningful efficiency metric is total token throughput per unit of power — tokens per second per megawatt — measured at the same output generation speed.

The same recipe scales past 3 trillion parameters: tally the memory needed for weights and per-request state, distribute layers across wafers, and chain the wafers into a pipeline. Each additional wafer contributes its own compute and on-chip memory.

Why it matters

Inference on GPU clusters is largely a memory-bandwidth problem: weights are streamed from HBM to the compute that consumes them. Cerebras's disclosure lays out the opposite bet — park every weight in SRAM, keep it there, and ship only the comparatively tiny activations between chips. The concrete numbers, from 16 GB per layer to 59 wafers and two-microsecond hops, give outsiders a first real look at how that architecture pencils out for a frontier-scale open model.

The post also shows how model architecture shapes serving costs. Kimi K3's mix of linear and dense attention matters directly here: linear layers carry no per-token KV state, which doubles how many fit per wafer and shrinks the cache footprint. As frontier models trend toward hybrid attention designs, memory behavior — not just parameter count — will help decide how cheaply they can be served. And with inference spend increasingly dominating AI economics, Cerebras's preferred yardstick, tokens per second per megawatt, is likely to become a standard basis for comparison.