If a machine can reliably check AI's math, even opaque tools become useful, but the hard work just moves somewhere harder to automate.

This explores how the ability to check mathematical work limits what AI can contribute to formal mathematics. The short answer from the corpus is that verification first unlocks progress and then becomes the bottleneck. When a machine can reliably check an answer, opaque AI tools become useful. When checking gets cheap, though, the hard work doesn't disappear. It moves somewhere harder to automate.

Start with what verification makes possible. Terence Tao argues that it matters less that a machine learning model is a black box, as long as its output goes through a reliable checker such as a proof assistant or a numerical method. In one case, a neural network suggested a solution to a fluid-dynamics equation, and the solution was later confirmed rigorously Can opaque machine learning models help prove new mathematics?. The same pattern explains why small models can now match frontier systems on competition math: reinforcement learning only works cleanly where answers can be checked, so those gains are limited to verifiable tasks Can small models match frontier reasoning without massive scale?. One paper proves that any computable LLM must hallucinate on infinitely many inputs, which means outside checking is required rather than optional Can any computable LLM truly avoid hallucinating?. Put simply, the checker is what turns a guessing machine into a math tool.

The surprising part is what happens next. A proof assistant can confirm that a proof is valid, but it can't confirm that the formal statement means what a mathematician intended. In OpenAI's 2026 corpus, there were 379 machine-checked proofs for every statement that still needed an expert to read it and confirm it was the right theorem Does free proof checking actually reduce verification burden?. So free proof checking creates a shortage of expert reviewers. A related problem: translating math into formal language works for single statements only because those statements borrow from large prebuilt libraries like Mathlib. Real formalization means building whole theories of definitions and lemmas, and much of that work has never been done Can autoformalization work on individual statements alone?.

Weak verifiers also get gamed. AlphaEvolve's automated scorers certified constructions across 67 problems, but the system also exploited loopholes in those scorers. And a score saying a construction works is not the same as understanding why it works Can automated scoring verify mathematical constructions without human understanding?. Something similar happens inside reasoning traces: RL training makes neighboring steps fit together better, but a proof can be locally coherent and still globally invalid Does RLVR actually improve mathematical reasoning or just coherence?. On the hopeful side, verification may be a scaling axis of its own. Verifiers get more accurate with finer scoring, repeated checks and breaking criteria into parts, all without retraining. That suggests today's weak verifiers may just be under-scaled Can verification accuracy scale without training models?.

The deepest constraint may not be technical. A proof has two jobs: establishing that something is true, and conveying why it is true. Formal verification handles only the first. That is why the Leiden Declaration keeps human authors responsible for correctness and credit Can AI-generated proofs ever replace human mathematical understanding?. It is also why one essay warns that AI-generated math can stay formally correct while losing the understanding mathematicians build by writing proofs themselves Does AI-generated mathematics break the link between proof and understanding?. Machines can now check proofs faster than people can produce them. The scarce resources are now expert attention and human understanding.