If an AI hands you a flawless proof using math you've never seen, have you actually understood anything?
Does a correct proof certify understanding if built from unfamiliar concepts?
This asks whether a proof that checks out as correct still shows that someone understands the mathematics, especially when the proof was built by an AI or from ideas the reader has never seen before.
This asks whether a proof that checks out as correct still shows that someone understands the mathematics, especially when an AI assembled it from ideas no human worked through. The corpus answers fairly clearly: no. Correctness and understanding used to come bundled together, and AI is pulling them apart. One essay argues that a proof traditionally did two jobs at once. It showed a claim was true, and the work of writing it was evidence that the mathematician had grasped why. When AI generates the proof, you can still verify it, but the understanding that writing used to produce never happens. The paper stays correct and loses its old role as proof of insight Does AI-generated mathematics break the link between proof and understanding?. The Leiden Declaration responds to exactly this gap. It makes human authors responsible for correctness and requires them to disclose AI use, because formal verification alone can't deliver both certainty and understanding Can AI-generated proofs ever replace human mathematical understanding?.
The most concrete case is the International Mathematical Olympiad. Expert graders confirmed that Gemini's proofs were complete and correct, but their review explicitly did not cover how the system got there What does correctness of outputs tell us about reasoning?. AlphaEvolve shows the same split across 67 problems. Automated scoring reliably certified its constructions, while explaining *why* they worked was a separate effort that succeeded only in many cases, not all Can automated scoring verify mathematical constructions without human understanding?. Terence Tao finds this split useful rather than alarming. An opaque tool can suggest real mathematics as long as a trusted checker, such as a proof assistant, confirms the result. Understanding can come afterward or be rebuilt by humans Can opaque machine learning models help prove new mathematics?.
Here's the less obvious part. Even a 'correct' certificate is weaker than it looks. Formal verification shows that a proof proves *something*, not necessarily the claim the mathematician intended Can LLM theorem provers tackle genuinely open-ended research problems?. And formalizing a single theorem quietly depends on a whole web of definitions borrowed from existing libraries Can autoformalization work on individual statements alone?. So 'built from unfamiliar concepts' cuts two ways. The ideas may be unfamiliar to the reader, and the framework doing the checking may be unfamiliar too.
Research on reasoning traces raises a further doubt about whether correct steps mean the system understood anything. Chain-of-thought examples with invalid logic work nearly as well as valid ones Does logical validity actually drive chain-of-thought gains?. Models trained on deliberately corrupted reasoning still reach right answers Do reasoning traces need to be semantically correct?. In both cases the reasoning seems to work as scaffolding rather than as meaning. Related work describes 'Potemkin understanding,' where a model explains a concept correctly, fails to apply it, and recognizes its own failure. That pattern suggests explanation and execution run separately inside the model Can LLMs understand concepts they cannot apply?. Interpretability research adds that models hold a patchwork of understanding, with deep principled circuits sitting alongside shallow shortcuts Do language models understand in fundamentally different ways?. A correct proof can't tell you which of these produced it.
The takeaway: a correct proof now certifies the *result*, not the *understanding*. Mathematics is responding by assigning understanding to a person, the human author who answers for the work, rather than reading it off the proof. Whether that holds up when no human can follow the concepts is an open question the corpus raises but doesn't settle.
Sources 11 notes
When AI generates proofs, verification remains possible but the human understanding built through writing practice is lost. Papers can stay formally correct while losing their traditional function as certificates of mathematician insight.
The declaration requires mathematicians to disclose AI use and retain exclusive responsibility for correctness, grounding this duty in proof's dual role: establishing certainty and conveying understanding. Formal verification alone cannot secure both goods.
Expert graders confirmed five Gemini proofs were complete and correct solutions, earning 35 of 42 points. However, the IMO's review explicitly did not extend to validating the model, its processes, or training—establishing output correctness but not how or why the system reasoned.
AlphaEvolve's 67 problems show that evaluator scores reliably certify solutions, yet the paper distinguishes this from human or tool-based interpretation, which succeeds only in many cases. Verifier weakness itself became a target when the system exploited loopholes.
Tao argues ML tools' opacity matters less than pairing them with reliable validators like proof assistants or numerical methods. He cites finite-time blowup for Boussinesq equations, where a neural network suggested solutions later verified through perturbation arguments.
Show all 11 sources
Current systems excel at isolated, well-defined proofs but cannot address truly open problems like Millennium Prize Problems. Many claimed successes rediscover existing results, and formal verification does not guarantee the proof addresses the intended mathematical claim.
Real formalization requires theory-level work: even one theorem needs a coherent web of axioms, definitions, and lemmas. Statement-level approaches only succeed by borrowing from prebuilt libraries like Mathlib, hiding the actual complexity involved.
Illogical chain-of-thought exemplars matched valid CoT performance on BIG-Bench Hard, showing that structural properties—not logical validity—drive the gains. The model learns the form of reasoning, not genuine inference.
Models trained on systematically irrelevant traces maintain solution accuracy and sometimes improve out-of-distribution generalization, suggesting traces function as computational scaffolding rather than meaningful reasoning steps.
Models can explain concepts accurately, fail to apply them, and recognize the failure—a triple pattern incompatible with human cognition. This indicates functionally disconnected explanation and execution pathways rather than simple knowledge gaps.
Mechanistic interpretability reveals conceptual understanding (features as directions), state-of-world understanding (factual connections), and principled understanding (compact circuits). Crucially, higher tiers coexist with lower-tier heuristics rather than replacing them, creating a patchwork of capabilities.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- From Solvers to Research: Large Language Model-Driven Formal Mathematics at the Research Frontier
- Verification abundance, adjudication scarcity: what happens to mathematical knowledge when proof checking becomes free
- Machine-Assisted Proof
- The crisis of AI-generated mathematics
- Mathematical methods and human thought in the age of AI
- The Debate Over Understanding in AI's Large Language Models
- Word Meanings in Transformer Language Models
- Is Chain-of-Thought Reasoning of LLMs a Mirage? A Data Distribution Lens