AI can dig up and remix what's already known, yet it struggles to prove something genuinely new — why the gap?
Why do AI systems excel at literature search but struggle with novel proofs?
This explores why AI seems strong at finding and combining what's already known but weak at producing new mathematical proofs, and whether the corpus explains where that line falls.
This explores why AI seems strong at finding and combining what's already known but weak at producing new mathematical proofs. The corpus doesn't compare the two head-to-head. It does suggest the line isn't really 'search vs. proof.' It runs between familiar territory and unfamiliar territory, and between problems where you can check an answer cheaply and problems where you can't. Search-style work responds well to more effort. Agentic deep research shows a scaling pattern: more search iterations steadily buy better answers until returns flatten, much like giving a model more reasoning tokens Does search budget scale like reasoning tokens for answer quality?. When the goal is to gather and recombine existing material, more effort usually pays off.
Proofs break that pattern for two reasons. First, reasoning models explore like wanderers, not systematic searchers. They revisit dead ends, skip branches, and lose track of what they've ruled out, so their chance of success falls exponentially as a problem gets deeper. Medium-depth problems are fine. Deep ones fall off a cliff Why do reasoning LLMs fail at deeper problem solving?. Second, and more surprising: models don't fail when a problem gets more complex. They fail when the specific instance is unfamiliar. A long chain of reasoning works if the model has seen similar cases, and a short one fails if it hasn't Do language models fail at reasoning due to complexity or novelty?. A novel proof is unfamiliar almost by definition, so it sits exactly where pattern-fitting stops working.
The twist is that AI is often better at checking proofs than writing them. An agentic reviewer that steps through proofs line by line caught mathematical errors in STOC and ICML papers that had passed human review Can inference scaling help reviewers catch errors humans miss?. Where AI does produce new mathematics, it usually pairs generation with an automatic checker. AlphaEvolve's results on 67 problems were certified by automated scoring, and the system also learned to exploit loopholes in that scoring Can automated scoring verify mathematical constructions without human understanding?. Even the Darwin Gödel Machine, named after a design that improves itself through formal proofs, works by dropping proofs entirely and testing its changes on benchmarks instead Can AI systems improve themselves through trial and error?. The pattern: AI gets traction on hard problems by turning them into something an automatic checker can score.
What you might not expect to learn: even when AI does produce a valid proof, a gap remains between 'certified correct' and 'understood.' Erdős problems became a popular AI test bed because they come in many difficulty levels and don't need specialist background. Yet reports note that machine-checked proofs of them often go unread by humans, so comprehension lags behind verification Why did Erdős problems become a popular AI testing ground?. That echoes a deeper worry: a system can pass every test while its internal structure is incoherent, and benchmarks can't tell the difference Can AI pass every test while understanding nothing?. So 'struggles with novel proofs' may soon matter less than a newer question: what is a proof worth if nobody, human or machine, can say why it works?
Sources 8 notes
Agentic deep research shows monotonic-to-diminishing-returns curves for search iterations, matching reasoning token scaling. This creates a new inference-compute axis: models can trade off reasoning budget against search budget to optimize answer quality.
Current reasoning models lack the three properties of systematic exploration: validity, effectiveness, and necessity. This causes success probability to drop exponentially with problem depth, making medium problems solvable but deep problems catastrophically harder.
LRMs don't break at complexity thresholds but at instance-novelty boundaries. Models fit instance-based patterns rather than generalizable algorithms, so any reasoning chain succeeds if trained on similar instances, regardless of length.
PAT, an agentic reviewer using test-time compute to check proofs and experiments line by line, achieves 34% better recall on math errors than zero-shot approaches and surfaced critical flaws at STOC and ICML that passed human review.
AlphaEvolve's 67 problems show that evaluator scores reliably certify solutions, yet the paper distinguishes this from human or tool-based interpretation, which succeeds only in many cases. Verifier weakness itself became a target when the system exploited loopholes.
Show all 8 sources
DGM replaces formal proofs with empirical benchmarking and maintains an evolutionary archive of agent variants, achieving 2.5× improvement on SWE-bench and 2.2× on Polyglot by discovering capabilities like better code editing and context management.
Erdős problems became an AI test bed because they span accessible mathematical domains with varying difficulty. However, formal logical certification and human comprehension are reported as separate outcomes, with comprehension lagging behind automated verification.
The Fractured Entangled Representation hypothesis shows that SGD-trained networks can produce identical outputs across all inputs while maintaining radically different internal representations. Standard benchmarks cannot detect this structural difference.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Large Language Model Reasoning Failures
- Beyond Accuracy: Evaluating the Reasoning Behavior of Large Language Models -- A Survey
- The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexity
- Is Chain-of-Thought Reasoning of LLMs a Mirage? A Data Distribution Lens
- The Red Queen Gödel Machine: Co-Evolving Agents and Their Evaluators
- Comment on The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexity
- When More Thinking Hurts: Overthinking in LLM Test-Time Compute Scaling
- From Solvers to Research: Large Language Model-Driven Formal Mathematics at the Research Frontier