· via dev.to (home feed)
Domyn's 1T-parameter MoE hits 1,293 tokens per second per GPU in JUPITER trials
Domyn reports trials of a 951.9-billion-parameter mixture-of-experts model on Europe's JUPITER supercomputer reaching a median 1,293 tokens per second per GPU, up from 90 at the start.

A technical post from Domyn, the Milan-based company leading the EUROPA consortium, describes mixture-of-experts training trials on JUPITER, Europe's first exascale supercomputer, in which throughput climbed from roughly 90 tokens per second per GPU at the start to a median of 1,293. According to a dev.to summary, the same write-up appeared on the PyTorch blog on November 12, 2026, and Domyn is explicit that the results come from trials rather than a completed pretraining run.
A near-trillion-parameter model on public hardware
EUROPA, formed by Domyn and two Fraunhofer institutes, was selected on June 19, 2026 by the European Commission as the winner of the EuroHPC Frontier AI Grand Challenge. The Commission's conditions include an open model of at least 400 billion parameters, coverage of all 24 official EU languages, and compliance with the EU AI Act.
JUPITER is hosted at Forschungszentrum Jülich, and each of its Booster nodes carries four NVIDIA GH200 Grace Hopper Superchips. The configuration Domyn reports totals 951.9 billion parameters, with 32.8 billion active per token, 384 routed experts using top-8 routing, and a pretraining sequence length of 65,536 tokens. Throughput was measured on 384 GH200 GPUs spread across 96 nodes per data-parallel replica, with 1,536 GPUs used for four replicas. At around 1,300 tokens per second per GPU, the company calculates that one replica processes about 43 billion tokens per day and that the four-replica setup handles roughly 150 billion.
Where the speedup came from
Domyn attributes the gains to a short list of engineering changes. Whole-expert sharding for the Muon optimizer lets each rank own complete expert matrices, so Newton-Schulz iterations run without any communication between ranks. Running Newton-Schulz on the GPU while parameters stay offloaded cut the optimizer step from 41.7 seconds to 2.2 seconds and lifted throughput from 441 to 597 tokens per second per GPU. Offloading activation checkpoints to the Grace CPU raised throughput from 370 to 508 tokens per second per GPU. Running the FSDP all-gather on the compute stream, meanwhile, took allocator retries from 384 down to zero.
Caveats in the small print
The post's own disclosures limit how far the headline number travels. Throughput figures are medians over steps 4 to 10, and the loss comparison covers only ten training steps. No model weights, release date or licence have been announced, and neither the training data nor its language mix is described. Domyn itself warns that spreading tokens evenly across four groups of 96 experts can hurt training quality.
Independent reporting cited by dev.to adds competitive context: Mistral AI released open weights for Mistral Large 4, a model of roughly one trillion parameters, on October 29, 2026, while Meta's Muse Spark Open stands at 816 billion parameters.
Why it matters
Europe's Frontier AI Grand Challenge rests on the premise that a shared public supercomputer can produce a frontier-scale open model. These trials are concrete evidence that a near-trillion-parameter MoE can run at high throughput on JUPITER, and utilization is the number that decides whether a 400-billion-plus-parameter model covering 24 languages is actually trainable within a sane budget and calendar. The engineering notes, from Muon expert sharding to checkpoint offload onto Grace CPUs, are transferable to other large runs on comparable hardware. But the limits are as instructive as the throughput: a ten-step loss comparison and undisclosed data leave questions of model quality, licensing and timing unanswered. Until a full run completes and weights ship under an open licence, this is a milestone in infrastructure engineering rather than a finished European answer to the trillion-parameter models already on the market.
- #mixture-of-experts
- #supercomputing
- #llm-training
- #european-union
- #pytorch