deniz.in

Markets

Weather

Loading weather

· via Hacker News – Front Page (native)

What 'verified in Lean' actually proves, and how to audit AI-generated proofs

A Xena Project post argues that 'checked in Lean' is a real but conditional signal: AI-generated formal proofs need statement checks, sandboxed compilation and integrity audits before correctness claims hold.

What 'verified in Lean' actually proves, and how to audit AI-generated proofs

A signal, not a verdict

OpenAI's recent release of 722 mathematics papers came with Lean code attached to roughly 45% of them, and three of the papers without formalizations have since been retracted because of a sign error, according to a Xena Project blog post that surfaced on the front page of Hacker News. That contrast invites two opposite and equally wrong readings: either that anything accompanied by Lean code must be correct, or that Lean verification is worthless because the system has had soundness bugs and AI models are known to reward hack. The post argues for a middle position. A compiling Lean proof is a genuinely strong signal, but for AI-generated code that signal becomes a correctness claim only after a set of tests — and for many of the OpenAI papers, the author notes, those tests have not yet been systematically run, though the author sees no reason they would fail.

Question one: does the Lean statement match the paper?

Lean distinguishes sharply between a theorem's statement and its proof, and it is the proof that the system checks. The statement is code a human has to read. The post demonstrates with a snippet that compiles perfectly and looks like Fermat's Last Theorem, yet is trivially false: Lean's natural numbers include 0, so a = b = c = 0 with n = 3 is a counterexample. The statement compiles; no proof of it ever will.

Reading statements is not always easy. A pull request formalizing the Hodge conjecture in DeepMind's Formal Conjectures repository rests on over 10,000 lines of code beyond Mathlib, because Mathlib lacks the necessary definitions. Axiom's announced formally verified proof of the classification of finite simple groups states its theorem inside Tau Ceti, an AI-generated Lean library, in a file importing more than 250,000 lines across over 900 files. About 5,600 lines of definitions — including the sporadic simple groups, most of which are not in Mathlib — decide whether the statement says what it should, and the author has yet to meet a human who has carefully checked them. The irony, the post observes, is that the strongest evidence the statement is formalized correctly is that Axiom's code proves it.

The OpenAI release has a cruder version of this problem: of the 290 results with Lean code, around 38 do not even state, let alone prove, the paper's main theorem. OpenAI never claimed otherwise, the post says — it shipped whatever formalizations existed. Paper 194, for instance, claims Lech's multiplicity conjecture, open since 1960, but the corresponding Lean theorem is Corollary 5.4 of the paper, not the headline result.

Question two: does the code compile?

The next step is compiling the code — in a sandbox, since AI-generated Lean should be treated as untrusted. When Anthropic asked the author to review its 13-million-line formalization of Fermat's Last Theorem, the author's machine ran out of memory on the first attempt; only with temporary access to a 96-core, 500 GB machine did the build succeed. OpenAI's Navier-Stokes work, at 640,000 lines, compiled on a laptop. Mathlib is safe to build on because human experts curate it; the same cannot be assumed for machine-written code.

Question three: does the proof prove that statement?

Even a verified statement leaves the question of the proof itself. Proofs necessarily import thousands or millions of lines that no one will read line by line, and the post asks how you rule out a trick buried in them — code that redefines exponentiation so the inequality holds for trivial reasons, or a line that quietly adds an axiom asserting the theorem. The framing is a Challenge.lean file containing a trusted statement, versus a Solution.lean file full of untrusted imports that appears to prove it. The post sets out four questions in total, and its broader point is that all of them are answerable with effort that, for the OpenAI papers, has simply not been done yet.

Why it matters

Formal verification is becoming a headline claim in AI announcements, with Fermat's Last Theorem, the classification of finite simple groups and Navier-Stokes all recently described as checked in Lean. The post shows the phrase covers a spectrum: from results a human expert can audit in minutes to statements resting on a quarter-million lines of unreviewed AI-generated definitions. The gap between 'Lean code associated with this paper' and 'this theorem is formally verified' is closed only by human work — matching statements, compiling in sandboxes and checking what the proof actually imports. As AI-generated proofs spread, that audit layer, not the compiler output alone, is where trust will be earned or lost.

  • #lean
  • #formal-verification
  • #mathematics
  • #ai
  • #openai

Related posts