deniz.in

Markets

Weather

Loading weather

· via dev.to (home feed)

Microsoft claims MAI Cyber 2 Flash beats GPT-6 Astra on security tasks at under 2% of the cost

Microsoft says its specialist cybersecurity model MAI Cyber 2 Flash beat GPT-6 Astra on the CyberGym-E2E benchmark while costing under 2% as much per task, though results come from specific offline evaluations.

Microsoft claims MAI Cyber 2 Flash beats GPT-6 Astra on security tasks at under 2% of the cost

Microsoft has announced MAI Cyber 2 Flash, a cybersecurity-focused model that the company says beat GPT-6 Astra on its CyberGym-E2E benchmark while costing less than 2% as much per task in offline evaluations. According to a dev.to report on the announcement, the accuracy gap was narrow — 92.3% versus 91.5% task success — but the economics were not, and Microsoft says its model got there using half the reasoning budget of its competitor.

What was measured

Microsoft pitches CyberGym-E2E as a benchmark grounded in real security work, covering tasks such as uncovering vulnerabilities, drafting fixes and investigating threats. For its comparison, Microsoft evaluated 813 tasks after excluding 92 of the benchmark's original 905, because those came from open-source projects and vulnerability reports that had been shared with training data. That exclusion is meaningful context: it lowers the chance that the scores reflect a model recognising material it saw during training, but it also means the headline figures describe the reduced 813-task set rather than the full benchmark.

Microsoft also reports favourable WebExploitBench results against GPT-5.6 Cyber at an equivalent reasoning budget, though the dev.to summary gives no numbers for that comparison. The company says more comparative results will follow in the coming weeks.

Where the cost claim comes from

The economics rest on listed token pricing. Microsoft lists MAI Cyber 2 Flash at $0.10 per million input tokens, $0.01 per million processing tokens and $0.50 per million output tokens, while GPT-6 Astra on Azure OpenAI is listed at $10, $1 and $50 for the same categories — a uniform 100-fold difference. Combined with the halved reasoning budget, that yields the sub-2% cost-per-task figure in Microsoft's offline evaluation.

As the dev.to write-up cautions, a cheaper token rate does not guarantee a cheaper bill, because real workloads vary in how many tokens they consume at each stage. The sub-2% number is best read as an evaluation-specific efficiency finding, not a pricing promise.

A model built for security work

Rather than a general-purpose system, MAI Cyber 2 Flash is purpose-built to locate vulnerabilities, recommend fixes and work through threat investigations at scale. Its context window of one million tokens is four times that of MAI Cyber 1.1 Flash, giving it room to reason over larger codebases and more evidence during multi-step investigations.

Microsoft additionally cites early Perception evaluations showing performance comparable to leading models on investigation-agent and security-detection authoring work, and says Insight Enterprises reportedly measured the MAI Cyber models at around 20% better token efficiency than some GPT-based competitors in independent tests.

Rollout into Microsoft's security products, including one referred to as Codename MDASH, is slated to begin in mid-December 2026. Exact availability, feature scope and customer access details were not specified.

Why it matters

The result is a concrete data point for the argument that narrow, specialist models can beat frontier generalists on value rather than raw capability. MAI Cyber 2 Flash did not dramatically outscore GPT-6 Astra — 0.8 percentage points is a margin narrow enough that evaluation methodology matters — but doing equivalent work at a sliver of the cost changes the arithmetic of running AI assistance routinely across code reviews and investigations, instead of saving it for exceptional cases.

For buyers, the useful metric becomes cost per finished task that a human can review, not price per token. That framing pushes evaluation toward an organisation's own code, security data and review standards, since a single vendor's offline benchmark — with roughly a tenth of tasks removed over contamination concerns — will not transfer automatically to every environment.

There is also a distribution angle. A specialist model embedded in the security products teams already use removes the friction of moving findings between a model and the tools where triage and response happen. If the December rollout delivers on that, the integration may end up mattering as much as the benchmark score.

  • #microsoft
  • #cybersecurity
  • #artificial-intelligence
  • #llm
  • #benchmarks

Related posts