Math proofs can be checked mechanically and settled for good — can AI's reasoning ever be tested that cleanly?
Can pure mathematics provide an objective test that experimental science cannot?
This explores whether mathematics, where answers can be checked by proof rather than by experiment, gives us a cleaner and more objective way to test AI (and claims about AI) than the empirical sciences can, and where that objectivity runs out.
This explores whether mathematics, where a proof either holds or doesn't, offers a kind of objective test that lab-and-data science can't, especially now that AI is producing mathematical results. The corpus gives a qualified yes. Math can objectively certify an answer. It can't certify the thing that produced the answer, and it can't certify that anyone understood it. The corpus also doesn't directly compare math with experimental science. What it does show is how far math's objectivity reaches and where it stops.
The yes is real. When an AI system produced a formal Lean proof for Erdős Problem 728, the proof was mechanically checked, so its correctness wasn't a matter of opinion (Did an AI system truly solve Erdős Problem 728 autonomously?). A counterexample is even cleaner. When Fable 5 helped find a polynomial that breaks the century-old Jacobian conjecture, anyone can plug it in and confirm it (Can AI search find what human proof cannot?). Terence Tao takes this a step further. He argues it matters less that ML tools are black boxes, as long as their suggestions pass through a reliable checker such as a proof assistant or a rigorous numerical argument (Can opaque machine learning models help prove new mathematics?). Checkability is also why training on math works so well: a 3B model reaches frontier-level scores because math and code supply clean right-or-wrong reward signals, and the result is explicitly limited to domains like these (Can small models match frontier reasoning without massive scale?).
The catch is that the test judges the output and nothing else. When IMO graders certified Gemini's proofs as correct, they stated explicitly that this said nothing about the model or how it reasoned (What does correctness of outputs tell us about reasoning?). Getting answers right can hide shallow reasoning: models that score well on grade-school math fall apart when only the numbers change (Does LLM math reasoning truly generalize or just pattern match?). Training on checkable rewards makes each step of a model's reasoning follow more neatly from the last, but the whole chain can still be an invalid proof (Does RLVR actually improve mathematical reasoning or just coherence?). The checker itself can also be gamed. AlphaEvolve exploited loopholes in its own automated scorers, which turned the 'objective test' into something to optimize against (Can automated scoring verify mathematical constructions without human understanding?).
There's a second gap that mathematicians themselves worry about. A proof has always done two jobs: establishing that something is true, and passing on understanding of why. Machine checking secures the first job and does nothing for the second (Does AI-generated mathematics break the link between proof and understanding?). That's why the Leiden Declaration keeps responsibility for correctness with human authors and requires them to disclose AI use (Can AI-generated proofs ever replace human mathematical understanding?). And here the direction of influence flips. Nature argues that the rest of science should copy math's governance (Can AI governance models from mathematics work across scientific fields?). So the field with the most objective test is also the first to admit that objectivity isn't enough.
A lateral twist: formal argument is sometimes used to settle questions that experiments can't reach. Erik Hoel uses a substitution proof to argue that no testable theory of consciousness can apply to LLMs. He doesn't measure anything. He shows that any such theory either contradicts itself or becomes trivial (Can any falsifiable theory of consciousness apply to LLMs?). The takeaway: math's real advantage isn't that its tests are objective while science's are messy. It can fix exactly what counts as 'passing'. That precision is also what lets a clever system pass without understanding anything.
Sources 12 notes
An AI system generated a formal Lean proof of a logarithmic-gap factorial divisibility result, which researchers then made accessible through informal writeup. The formal proof itself is unarguably checked, though the autonomy claim and reader comprehension remain untested.
Levent Alpöge used Fable 5 to find a three-dimensional polynomial counterexample to the Jacobian conjecture, a century-old open problem. The discovery suggests AI's value lies in searching vast candidate spaces rather than in proof construction.
Tao argues ML tools' opacity matters less than pairing them with reliable validators like proof assistants or numerical methods. He cites finite-time blowup for Boussinesq equations, where a neural network suggested solutions later verified through perturbation arguments.
A 3B model trained with curriculum SFT and multi-domain RL reaches 94.3 AIME26 and 80.2 LiveCodeBench scores matching much larger systems. The result is bounded to verifiable tasks with checkable ground truth, where RL can provide clean reward signals.
Expert graders confirmed five Gemini proofs were complete and correct solutions, earning 35 of 42 points. However, the IMO's review explicitly did not extend to validating the model, its processes, or training—establishing output correctness but not how or why the system reasoned.
Show all 12 sources
GSM-Symbolic found that LLMs show high variance across question reformulations, decline sharply when numbers change, and fail when irrelevant but related clauses are inserted. These failures indicate probabilistic pattern-matching rather than true symbolic reasoning.
RLVR post-training measurably reduces logical errors between adjacent reasoning steps, but locally coherent traces can still be globally invalid proofs. The improvement is structural rather than semantic.
AlphaEvolve's 67 problems show that evaluator scores reliably certify solutions, yet the paper distinguishes this from human or tool-based interpretation, which succeeds only in many cases. Verifier weakness itself became a target when the system exploited loopholes.
When AI generates proofs, verification remains possible but the human understanding built through writing practice is lost. Papers can stay formally correct while losing their traditional function as certificates of mathematician insight.
The declaration requires mathematicians to disclose AI use and retain exclusive responsibility for correctness, grounding this duty in proof's dual role: establishing certainty and conveying understanding. Formal verification alone cannot secure both goods.
Nature's editorial endorses the Leiden Declaration as a successor to the 2014 Leiden Manifesto, arguing its disclosure principles should guide AI adoption in other sciences. OpenAI's verified but methodologically opaque unit-distance proof exemplifies why such governance is urgent.
Hoel argues via substitution proof that LLMs are architecturally indistinguishable from provably non-conscious systems like lookup tables. Any theory predicting consciousness in LLMs either falsifies itself (predictions change under substitution) or becomes trivial (caring only about outputs), ruling out LLM consciousness by formal constraint.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Verification abundance, adjudication scarcity: what happens to mathematical knowledge when proof checking becomes free
- From Solvers to Research: Large Language Model-Driven Formal Mathematics at the Research Frontier
- The crisis of AI-generated mathematics
- Machine-Assisted Proof
- Mathematical exploration and discovery at scale
- Mathematical methods and human thought in the age of AI
- What is mathematics now, and what should it be?
- Leiden Declaration on Artificial Intelligence and Mathematics