Yandex open-sources AliceAI-Foundation-80B-A3B-Base, an 80B MoE trained from scratch

What Yandex released

Yandex has open-sourced AliceAI-Foundation-80B-A3B-Base, a language model published on Hugging Face under Apache 2.0, which permits commercial use. According to a technical write-up on dev.to, the model was trained from scratch on roughly 18 trillion tokens of Russian and English text, with no weights borrowed from existing families such as Qwen or Llama.

It is a base model, not an instruction-tuned assistant: it continues text rather than chatting, and Yandex itself labels the release experimental. The company positions it as a testbed for the architecture of an upcoming unified reasoning model that will power agentic features in its Alice AI assistant.

Compared with Yandex's previous flagship, the 235B-parameter Alice AI LLM released in October 2025, the new model is nearly three times smaller overall and about seven times smaller in active parameters. According to the technical report cited by dev.to, it nevertheless scores higher than its predecessor on factual knowledge, mathematics, code and long-context tasks.

Sparse experts plus linear attention

The model is a mixture-of-experts design with 48 layers. Each token activates 10 of 512 routed experts plus one shared expert that is always engaged, so the model carries 80B parameters in total while spending roughly the per-token compute of a 3B-parameter model.

The attention stack is the more unusual half of the design. Three out of every four layers use Kimi Delta Attention, a linear-attention variant that folds sequence history into a fixed-size state instead of storing a key-value entry for every previous token; the remaining layers are standard gated attention. This hybrid, according to the dev.to analysis, is what allows a 262,144-token context window without the KV-cache memory growth that makes long inputs expensive on conventional transformers.

Where it wins and where it doesn't

Figures from the official model card, as reported by dev.to, compare base versions against Qwen3.5 35B-A3B, Nemotron-3 Super 120B-A12B and DeepSeek-V4 Flash 284B-A13B.

On anything requiring knowledge of the Russian-speaking world the model leads comfortably, beating competitors with up to four times its active parameters: 86.5 on WikiWebFacts versus 83.2 for DeepSeek-V4 Flash, 74.2 on EduBench Russian against a next-best 67.7, and 49.6 on ExpertFactsQA Law against 40.5. Mathematics is also strong, with 91.1 on MATH-500 where no competitor exceeds 84.8, and 96.9 on HMMT February 2026 at pass@32 versus 87.9 for Qwen3.5. LiveCodeBench v5-6 is essentially a tie at 50.5.

English-centric general knowledge is where the parameter ceiling shows. TriviaQA comes in at 79.0 against 89.8 for Nemotron and 89.4 for DeepSeek, MMLU-Pro at 66.8 trails Nemotron's 69.9, BigCodeBench at 48.3 is marginally behind, and LongMemEval at 128K context sits below DeepSeek's 68.0. The dev.to author's reading is straightforward: 80B parameters cannot encode as much world knowledge as 284B.

Can you run it locally?

The 3B active-parameter figure is misleading for hardware planning, since all 80B parameters must sit in memory. The dev.to post puts this at roughly 160 GB in bf16 and, by the author's own estimate, around 45 GB at 4-bit quantization — in theory reachable on a 64 GB Mac, where MoE models run efficiently because only a small slice of weights is read per token.

In practice that is not yet realistic. Official recipes cover transformers and vLLM on NVIDIA GPUs, with the provided example using four of them; the Kimi Delta Attention layers depend on flash-linear-attention kernels, and no GGUF or MLX builds existed on release day. The author notes that non-standard attention typically delays llama.cpp support by weeks.

Caveats

Beyond being a base model that needs instruction tuning before it is useful as an assistant, two of the most striking benchmark results, WikiWebFacts and HardMultiQA, are Yandex's own. Their evaluation protocols were published alongside the weights so anyone can re-run them, but until third parties do, the post advises weighting independent benchmarks more heavily. It is also, plainly, not a frontier model: a compact model winning on data and architecture rather than raw scale.

Why it matters

Genuinely independent large models trained from scratch remain rare, and permissively licensed ones with an open technical report and reproducible benchmarks rarer still. For anyone building Russian-language products — legal, education and reference applications especially, where the model's lead is widest — this is a usable foundation rather than a curiosity. The architecture is equally notable: a 512-expert MoE with a 3:1 mix of linear and full attention delivering a 262K context at 3B active parameters is a concrete open data point on how vendors are engineering for long-context, agentic workloads, and it doubles as a preview of the design Yandex intends to ship inside its Alice assistant.