ByteDance's Axion 1.0 open agent model trained on just 18K examples

What was released

On November 2, 2026, a team from ByteDance, working with researchers from Duke University, the University of Maryland and the University at Buffalo, released Axion 1.0, an agent foundation model available in 35B and 122B parameter sizes. According to a write-up published on dev.to, the distinguishing feature is the budget: 18,363 training trajectories in total, roughly 1,000 GPU-hours for the 122B variant, and a recipe that skips mid-training, direct preference optimization and on-policy distillation entirely. Everything, from training code to datasets to weights for both sizes, ships under Apache 2.0.

Both variants use a Mixture-of-Experts architecture and were fine-tuned with a 128K token context window.

A three-stage recipe

Training runs in three stages: two rounds of supervised fine-tuning followed by reinforcement learning.

The first SFT stage builds tool-free scientific reasoning. Seed problems are synthesized from public sources and then hardened against the base model's own successful solution strategies, using transformations such as composition and parameterization. The result is 10,819 trajectories tuned to be difficult for the model at the moment they are generated, an approach the team describes as model-aware data construction, with no human labelers deciding what counts as hard.

The second stage introduces tool-dependent reasoning. Tasks are grounded in real databases, namely the NASA Exoplanet Archive, OpenAlex and UniProt, and reference answers are computed programmatically before the question is even written, giving every example a machine-checkable ground truth. This stage adds 7,544 trajectories, about 312 million tokens, and teaches the model to fill information gaps through retrieval.

The final stage applies Group Relative Policy Optimization over long-horizon agent trajectories, run asynchronously so rollout workers interact with tools and environments while policy updates proceed in parallel. Training operates at the session level to cope with the long interaction histories that agentic tasks produce.

Infrastructure built for consistency

The dev.to post highlights two mechanisms aimed at reproducibility. Token-In-Token-Out keeps token IDs stable across the pipeline, so tokenization differences between rollout and training cannot silently corrupt updates. Rollout Routing Replay addresses a Mixture-of-Experts quirk: expert routing during rollout can diverge from routing during the training forward pass, and R3 replays the rollout routing decisions so gradient updates target the same experts that generated the behavior in the first place. The stack combines FSDP with expert and context parallelism, and uses swift for SFT and verl for RL.

Reported benchmark results

Evaluation covered six benchmarks spanning scientific reasoning, information seeking and biological data analysis: Humanity's Last Exam, BioMysteryBench, SEAL-0, HiPhO, and the Olympiad and Research splits of FrontierScience.

The 122B model scores 59.0 on HLE and 66.7 on FrontierScience Research, a split that tests the ability to combine findings from multiple scientific papers. The 35B model scores 52.2 on HLE and 62.2 on SEAL-0. The post reports the smaller model matching Gemini 3.1 Pro on most tasks at a fraction of the running cost, though the Gemini figures were evaluated by the Axion team itself rather than a neutral party. The comparison agent Agents-A1 scored 47.6 on HLE.

Why it matters

Axion 1.0 is a data point against the assumption that capable agents require sprawling pipelines and large proprietary datasets. If around 18,000 carefully constructed, programmatically verifiable trajectories plus roughly 1,000 GPU-hours reach this performance tier, task design and data quality may matter more than sheer volume, which is encouraging for teams without massive compute budgets.

The release is also unusually complete: data construction and filtering code, SFT and asynchronous RL code, an evaluation harness, the dataset on Hugging Face, and weights for both sizes. Because the recipe is modular, stage one can be swapped for domain-specific reasoning data and stage two grounded in private databases, making the checkpoints a practical base for further fine-tuning. More broadly, it is another sign that agent training is shifting from bespoke engineering toward documented, reproducible recipes.