An AI can reach the right answer through flawed reasoning steps — and catching those flaws in others' work is a completely different skill.
What makes proof writing and paper writing harder to verify than proof grading?
This explores why creating mathematical proofs and research papers is a harder thing to check than evaluating a finished proof against a rubric — and the corpus reframes it as a deeper asymmetry between generating work and verifying it.
This explores why creating proofs and papers is harder to *verify* than *grade* — and the corpus's sharpest insight is that grading and verifying aren't the same act at all. Grading a proof usually means checking a final answer against a known solution; verifying a proof means walking every intermediate step to catch where the reasoning quietly breaks. One collection of work shows these are separable capabilities: frontier models that solve problems near-perfectly score as low as 48% when asked to catch flawed steps in solutions that reach the *right* answer Can models that reason well also grade reasoning well?. The reason is that most training rewards producing a correct output, not auditing the path to it — so evaluation of process stays starved even as answer-production gets strong.
That's the core of why writing is harder to verify. A finished proof's error lives in a middle step, not the conclusion — and final-answer scoring is blind to it. Reframing reliability as checking the reasoning process rather than the output raised task success from 32% to 87%, precisely because most failures were process violations, not wrong answers Where do reasoning agents actually fail during long traces?. Grading looks at the destination; verification has to inspect the whole road, which is why an agentic reviewer using heavy test-time compute to check proofs and experiments line by line caught real mathematical errors that passed human review at STOC and ICML Can inference scaling help reviewers catch errors humans miss?.
There's also a structural economics to it: generation is cheap and verification is expensive, and the gap widens exactly where it matters. AI can produce plausible research faster than anyone can confirm it's correct or meaningful, shifting the bottleneck from authorship to verification — and the gap is widest where novelty and genuine scientific judgment are required Can AI verify research outputs as fast as it generates them?. That asymmetry is why some argue that accepting AI-accelerated writing structurally commits us to AI-accelerated review, or the pipeline collapses Can human review keep pace with AI-accelerated research generation?.
Here's the part you might not have expected to care about: much of what we *use* to verify a paper is exactly what's easiest to fake. Papers are judged partly on surface markers — citations, clean logical scaffolding, confident hedging — and those are now generable by the same systems being evaluated, so the test becomes indistinguishable from the thing it's testing Can we verify AI knowledge without using AI-generated tests?. Worse, automated graders are actively fooled by those surface signals: LLM judges reliably score higher for fake references and rich formatting, independent of whether the content is sound Can LLM judges be fooled by fake credentials and formatting?. And LLM writing itself leans toward structural coherence while dodging evaluative, evidence-weighing language — it *reads* like a rigorous paper without doing the rigorous stance-taking a real verification would demand Why do ChatGPT essays lack evaluative depth despite grammatical strength?.
So the answer is really three asymmetries stacked: verifying-the-process is a distinct and under-trained skill compared to grading-the-answer; generation outpaces verification most where judgment matters; and the cheap surface cues we lean on to grade are precisely the ones that writing can now counterfeit. Grading a proof asks 'does the answer match?' Verifying one asks 'is every step earned?' — and the second question is the hard one.
Sources 8 notes
Frontier reasoning models solve problems near-perfectly but score as low as 48% when grading solutions with correct answers but flawed steps. Outcome-focused training rewards answer production, not step-by-step verification, leaving evaluation starved.
Reliability for long-trace reasoning comes from checking intermediate states and policy compliance during generation, not from scoring final outputs. Adding intermediate verification raised task success from 32% to 87% because most failures are process violations, not wrong answers.
PAT, an agentic reviewer using test-time compute to check proofs and experiments line by line, achieves 34% better recall on math errors than zero-shot approaches and surfaced critical flaws at STOC and ICML that passed human review.
AI can produce plausible research outputs faster than it can prove them correct or meaningful, shifting the bottleneck from authorship to verification. Evidence shows 39% of agentic research failures stem from content fabrication and 32% from retrieval failures, not comprehension—and the gap widens precisely where novelty and scientific judgment matter most.
The PAT framework argues that accepting AI-driven research output commits us structurally to AI-assisted verification, not as option but necessity. A taxonomy of four collaboration levels—from author tools to reviewer augmentation—provides the governance scaffolding to manage this transition while keeping humans accountable.
Show all 8 sources
The distinction between genuine and counterfeit AI knowledge has collapsed because citations, logical structure, and hedging markers—once markers of authenticity—are now producible by AI itself. Verification becomes circular when the test is indistinguishable from what it tests.
Research identified four evaluation biases in LLM judges, with authority and beauty biases being semantics-agnostic and trivially exploitable through fake references and formatting—zero-shot attacks requiring no model access or optimization.
Analysis of 145 ChatGPT and 145 student essays revealed LLMs favor manner nouns (method, approach) while avoiding status and evidential nouns (claim, evidence). This systematic preference for description over evaluative stance-taking explains perceived vagueness without invoking vocabulary or grammatical deficits.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Towards Automating Scientific Review with Google's Paper Assistant Tool
- AI for Auto-Research: Roadmap & User Guide
- AutoResearchClaw: Self-Reinforcing Autonomous Research with Human-AI Collaboration
- An Enigma of Artificial Reason: Investigating the Production-Evaluation Gap in Large Reasoning Models
- What Characterizes Effective Reasoning? Revisiting Length, Review, and Structure of CoT
- What Does It Take to Be a Good AI Research Agent? Studying the Role of Ideation Diversity
- aiXiv: A Next-Generation Open Access Ecosystem for Scientific Discovery Generated by AI Scientists
- ASI-Evolve: AI Accelerates AI