deniz.in

Markets

Weather

Loading weather

· via dev.to (home feed)

Anthropic's FlashProve generates zero-knowledge proofs of LLM inference at 110 tokens per second

Anthropic's Frontier Red Team has open-sourced FlashProve, a zkSNARK prototype that proves a committed set of model weights produced a given response, verifiable in seconds on an ordinary CPU.

Anthropic's FlashProve generates zero-knowledge proofs of LLM inference at 110 tokens per second

What FlashProve does

When an application calls a model over an API, the model identifier that comes back with the tokens is an assertion from the provider. Nothing in the response rules out a router quietly substituting a cheaper model, a tampered inference server, or a stronger model standing in for an approved one. A write-up on dev.to summarises how Anthropic's Frontier Red Team, working with researchers from EPFL, StarkWare, MIT and others, published an open-source prototype called FlashProve on 2 November 2026 to make that claim checkable.

The idea is simple in outline: a provider first cryptographically commits to one exact set of weights, publishing a short fingerprint while keeping the weights hidden. Each response can then carry a proof that those committed weights produced exactly those output tokens for a given prompt. According to the dev.to post, the verifier needs no GPU and learns nothing about the weights.

How it works

FlashProve is built on a zkSNARK, a short certificate that a computation ran correctly without revealing anything beyond that fact, and one that is far cheaper to check than to redo. Under the hood, as the post describes, it follows a familiar recipe: an interactive oracle proof compiled into a cryptographic proof via hash functions, with the construction also offering post-quantum security.

It runs in two phases. Offline, the provider commits to the weights, which takes about 39 minutes per model on a single H100; the commitment covers roughly 16 million requests before renewal, and the client downloads about 6.5 MB of commitment data. Online, the prover emits a proof for each response that anyone holding the commitment, the prompt and the reply can verify.

Three engineering choices account for the speed. Because Maverick is a mixture-of-experts model, only about 17 billion of its 400 billion parameters are active per token, so prover work scales with the active experts while the proof still hides which ones ran. Proving happens after the fact and in batches, so the weight commitment is opened once per batch rather than per request; a batch of 24 requests took about 223 seconds. And weights are streamed in segments from host memory, so a single 80 GB GPU plus about 290 GB of host memory is enough.

The numbers

For Llama 4 Maverick, a 400-billion-parameter mixture-of-experts model with 4-bit weights, one H100 generates proofs at roughly 110 tokens per second. Each proof is about 19 MB per 1,024-token request, and checking takes about 17 seconds on a 24-thread CPU. The cost is roughly 70-95 times the GPU time of serving the same requests with vLLM, which the post judges too expensive for proving every token but workable for sampling traffic or high-stakes outputs such as security scans or evaluation runs.

The comparison with earlier zkML work shows how quickly the field has moved: ZKML proved GPT-2, at 117 million parameters, at about 159 seconds per token; zkLLM in 2024 handled LLaMA-2-13B at about 2.5 tokens per second; VerInf proved the same Maverick model at about 0.02 tokens per second, with 93.6 GB proofs that took nearly a day to verify. A commercial system called Attestable reached 53-77 tokens per second on Gemma 4 31B but is not open source. Anthropic's takeaway, as relayed by the dev.to author, is that the overhead over native computation is about two orders of magnitude, lower than previously expected.

A verifier worth trusting

Cryptographic code is brittle, so the team formalised completeness, soundness and zero-knowledge in the Lean proof assistant and proved them in the random oracle model. They ship a fast C++ verifier for daily use alongside a slower Lean verifier that provably accepts only what the soundness theorem allows. Two gaps remain, and the write-up states them plainly: humans still have to audit that the formal statements match the intent, and there is no formal link between the fast code and the Lean specification.

Limitations

The proof covers an exact integer version of the weights under greedy decoding; sampled decoding or floating-point implementations can diverge and no longer match the proved computation. Contexts are capped at 1,024 tokens, far short of real chat or agent sessions. Only mixture-of-experts models are supported, and the architecture is not hidden. Timing side channels are out of scope. It is a research prototype, and Anthropic notes it already met its Frontier Safety Roadmap goal through a separate approach, signing traffic and recomputing on a trusted server, and has since renamed the goal "inference verification".

Why it matters

Model identity in an API is today a matter of trust, and that touches routing audits, evaluations before and after release, tamper detection, provenance claims and regulators who may want evidence that approved models are being served. FlashProve shows that the question of which weights produced a piece of text can become a cheaply checkable property even at frontier scale. The overhead is still too high for routine use, but the design lesson generalises: put the guarantee in the small, formally verified component and require trust in nothing else. For engineers, the practical steps now are logging prompts, exact output tokens, model identifiers and decoding settings per request, and pinning greedy decoding wherever provenance matters.

  • #anthropic
  • #zero-knowledge-proofs
  • #llm
  • #cryptography
  • #open-source

Related posts