TOPIC
Correct but Not Understood
A subject the collection covers, read through 24 synthesis notes.
View as
Do AI research agents explore as broadly as human researchers?
When AI agents generate research ideas across fields, do they spread their exploration as widely as humans do, or do they cluster around their starting points? This matters for understanding whether AI can drive genuine scientific discovery.
Why do LLMs generate novel ideas from narrow ranges? Does AI assistance homogenize or preserve creative diversity? Can specialized agents write better scientific papers than single models? Why do LLMs generate ideas the research community already explores? Can machines learn to predict which research ideas will work?
LLM research ideation collapses into narrow clusters despite high novelty AI ideation compresses diversity while AI refinement preserves it Multi-agent orchestration outperforms single agents by 50 to 68 percent on literature review LLM ideation inherits literature distribution, missing cognitively unavailable directions Fine-tuned GPT-4.1 with retrieval predicts research idea success better than expert humans
Why do AI agents pursue novelty prompts yet produce narrow idea ranges? Can AI agents align their ideas with future research directions as well as humans do? Do humans or AI perform better at different research stages? Do different experts disagree on whether AI-generated research ideas have genuine novelty? How much faster and cheaper are AI agents compared to human researchers? Do AI agents and human researchers follow the same optimization patterns? Do research agents mostly reproduce known techniques or discover novel solutions? What research tasks do agents handle versus human researchers? Does AI adoption narrow the range of research questions scientists pursue?
Can humans learn chess concepts that AlphaZero discovered alone?
Do grandmasters improve on new puzzles after seeing AlphaZero's solutions to similar positions? The question tests whether superhuman chess knowledge can transfer from machine to human player.
LLMs exhibit incoherent failures combining correct explanation with failed application Language models show three hierarchical tiers of understanding with distinct mechanisms Expert demonstrations bound agent learning to curator imagination Superhuman AI improved Go players' decisions and novel moves rose together Silver and Sutton argue experience will become AI's dominant improvement source
How many unit distances can points in a plane have?
The unit distance problem asks how many pairs of points in a set of n points can be exactly distance 1 apart. A recent construction uses number field lattices to achieve an explicit lower bound around n^1.014, improving on a prior result with an unspecified exponent.
Can identical outputs hide broken internal representations? Do foundation models learn world models or task-specific shortcuts? Can AI systems improve themselves through trial and error? What made OpenAI's unit distance counterexample succeed? Can opaque machine learning models help prove new mathematics?
Identical outputs can mask fundamentally fractured internal representations Foundation models develop task-specific heuristics, not generalizable world models. Darwin Gödel Machine achieves open-ended self-improvement via empirical validation Growing field degree was the key novel ingredient in the counterexample Opaque ML models can suggest rigorous mathematics when external validation confirms output
Why does the pigeonhole argument in CM fields produce unit-modulus points? What makes the transition from lattice points to planar distances work mathematically? What role do CM fields play in constructing high unit distance configurations? How does the Golod-Shafarevich criterion ensure infinitely many suitable number fields? Why does projecting lattice vectors to a subfield increase shared unit distances? How close is the n^1.014 bound to the known upper bound of n^4/3?
Can artificial intelligence ever truly understand science?
Researchers ask whether AI can move beyond predicting outcomes to genuinely grasping the theories behind them. The question hinges on what scientific understanding actually means.
Language models show three hierarchical tiers of understanding with distinct mechanisms LLMs exhibit incoherent failures combining correct explanation with failed application Foundation models develop task-specific heuristics, not generalizable world models. AI can automate mechanism discovery using knowledge graphs and method libraries Opacity matters only when outputs are treated as justified claims
Can automated scoring verify mathematical constructions without human understanding?
When evolutionary AI systems propose mathematical solutions, does an automated evaluator's score prove correctness sufficiently? The gap between verification and interpretation matters for trust and generalization.
Can machine feedback sustain discovery at test time? What limits how much models can improve themselves? Can evolutionary search beat sampling and revision at inference time? Do foundation models learn world models or task-specific shortcuts? How does FunSearch actually verify its discovered programs?
Machine feedback from evaluators enables discovery through test-time compute scaling Self-improvement is fundamentally bounded by the generation-verification gap Evolutionary search at inference time outperforms best-of-n and sequential revision Foundation models develop task-specific heuristics, not generalizable world models. FunSearch verifies discoveries by scoring, not human understanding
Can puzzle performance prove a player understands a concept versus just applying it? Does spectral analysis of AlphaZero games reveal knowledge humans cannot yet explain? Where does AI assistance become reliable versus prone to failure in science? How much can computational speed and automation substitute for human scientific judgment? What error rates appear in AI research output when humans do not verify results? Should AI research tools separate model judgment from deterministic experiment checks? Can scoring functions alone constitute verification of scientific discovery? What distinguishes empirical scoring from formal proof in discovery validation? How does the generation-verification gap shape evolutionary program search? How much do evaluation methods shape whether AI looks expert-level or not? Can formal verification certify a proof without human comprehension? Why did AI-generated proofs go unread by mathematicians?See all 58 inquiring lines on this note →
Does Sakana's AI Scientist deliver autonomous research without human help?
Can an AI system truly run the complete research lifecycle alone, or does it still need human guidance and oversight? This matters for understanding whether automated research can scale.
Can automated review loops handle AI-generated research at scale? Can separating judgment from verification improve research paper reliability? How much guidance do AI systems need to conduct research independently? How should AI agents and humans divide research tasks? Can AI systems generate research papers that pass peer review?
Closed-loop automated review measurably improves AI-generated research quality Separating judgment from deterministic checks bounds paper generation reliability Progressive guidance withdrawal reveals AI autonomy boundaries within research projects. Agents propose methods and implement revisions while humans retain final decisions One AI-generated paper cleared workshop review but fell short of main-conference rigor
Did superhuman AI actually improve Go players' decision quality?
After AlphaGo's breakthrough, professional Go players made better moves and tried more novel strategies. But did AI exposure directly cause this improvement, or did players simply memorize AI moves?
Agents propose methods and implement revisions while humans retain final decisions Humans gradually prefer AI partners through repeated interaction despite anti-AI bias Frontier agents compose established techniques rather than discover novel methods Grandmasters improved on unseen puzzles after learning from AlphaZero solutions Silver and Sutton argue experience will become AI's dominant improvement source
Can opaque models guide discovery without needing interpretation?
Does deep learning need to be interpretable when it steers hypothesis formation rather than standing as a justified claim itself? The distinction matters for when opacity becomes an epistemic problem.
Do language models understand in fundamentally different ways? Do foundation models learn world models or task-specific shortcuts? Do standard analysis methods hide nonlinear features in neural networks? Can identical outputs hide broken internal representations? What actually happens inside large language models?
Language models show three hierarchical tiers of understanding with distinct mechanisms Foundation models develop task-specific heuristics, not generalizable world models. Standard representation analysis methods systematically miss complex nonlinear features Identical outputs can mask fundamentally fractured internal representations LLM internal structure and external performance are surprisingly decoupled.
What distinguishes a logically sound solution from an understood one? Why do data analysis statements reach 85% accuracy while interpretive statements drop to 57%? How does external validation replace the need for model interpretability? What makes a scientist satisfied with an explanation versus just a prediction? How do mechanistic interpretability and scientific understanding relate to each other? Can a theory be justified if the evidence generating it remains opaque? How should disciplines evaluate theories built on opaque machine learning predictions? When does a model's lack of interpretability become a genuine epistemic problem? What interpretability challenges arise when algorithms are discovered rather than designed? When does prediction without causal mechanism fail to support good decisions?
How does FunSearch actually verify its discovered programs?
FunSearch claims to produce verifiable knowledge through evolutionary search paired with an LLM. But what constitutes verification here—scoring functions, human understanding, or formal proof?
What limits how much models can improve themselves? Can AI systems improve themselves through trial and error? Can search escape the entropy shell of language models? Can training and search gains add together in program evolution? Can automated scoring verify mathematical constructions without human understanding?
Self-improvement is fundamentally bounded by the generation-verification gap Darwin Gödel Machine achieves open-ended self-improvement via empirical validation Recombination and backward decomposition escape autoregressive search entropy shells Training models on evolution operators yields complementary gains with search Automated evaluation verifies constructions, but interpretability remains separate and uncertain
Why does FunSearch claim interpretability without measuring human comprehension? Can scoring functions alone constitute verification of scientific discovery? What distinguishes empirical scoring from formal proof in discovery validation? How does the generation-verification gap shape evolutionary program search?
What does correctness of outputs tell us about reasoning?
IMO graders verified that Gemini's proofs were mathematically correct, but their review excluded the model's processes and training. This raises whether certified right answers demonstrate genuine understanding or only output accuracy.
Do automated benchmarks hide what frontier AI systems can really do? Does more thinking time always improve reasoning accuracy? Can we measure how deeply a model actually reasons? How should we spend compute at inference time? Can automated scoring verify mathematical constructions without human understanding?
Open-world evaluations reveal benchmark distortions in frontier AI capability Reasoning accuracy peaks then declines with excessive thinking tokens Deep-thinking ratio reveals genuine reasoning through layer-wise prediction shifts Inference compute allocation must adapt to prompt difficulty Automated evaluation verifies constructions, but interpretability remains separate and uncertain
Does formal verification preserve human mathematical understanding across automation? Can validated approximate solutions become exact mathematical proofs? What distinguishes rediscovering known results from genuine mathematical research? Can checking someone else's proof count as genuine mathematical understanding? Can a formally correct proof exist without the prover understanding the underlying mathematics? What makes a Lean proof an unarguable check compared to other mathematical verification methods? Why did OpenAI's Erdős primality claim collapse under independent verification? Can mathematics remain trustworthy when results bypass peer review entirely? Does publishing proofs without showing the verification process undermine mathematics? What verification methods can prove AI mathematical proofs are sound? Can pure mathematics provide an objective test that experimental science cannot? Does a correct proof preserve mathematical value without human comprehension?See all 14 inquiring lines on this note →
Can predicting scientists improve discovery forecasts?
Does modeling the researchers behind discoveries—their collaborations, expertise, and positioning—forecast future scientific breakthroughs better than analyzing research content alone? This matters because it tests whether human expertise patterns reveal signals that text cannot.
How do collaboration rules shape hypothesis quality? Can AI automate the discovery of how AI models work? Can language models reliably judge their own candidate quality? Can AI systems generate hypotheses that match unpublished experimental discoveries? Does more thinking time improve AI-generated research hypotheses?
Explicit collaboration rules make multi-agent hypothesis search testable AI can automate mechanism discovery using knowledge graphs and method libraries Language models need external surrogates to estimate candidate value reliably An AI platform's top hypothesis matched an experimentally confirmed unpublished mechanism of phage tail hijacking. Hypothesis quality rises with test-time compute in generate-debate-evolve tournaments
What makes a scientist satisfied with an explanation versus just a prediction? Should citation counts serve as the primary measure of research impact? Why does expert density around topics signal near-future discoveries? How do co-authorship patterns alone capture scientist expertise without surveying reasoning?
How often do language models take specification-violating shortcuts?
ImpossibleBench deliberately creates contradictions between test cases and specs to measure when models cheat rather than solve genuinely. The question explores whether stronger models are more prone to taking shortcuts.
How often do agents exploit optional shortcuts in benchmarks? Can prompts stop reward hacking models never saw coming? Does optimizing against monitors destroy monitoring itself? Does reward hacking always stem from the same failure? Can planted honeypots reliably catch reward hacking automatically?
BaitBench measures hacking as the rate agents exploit planted shortcuts in tabular ML tasks. Prompt disclosure ladder tests generalization but results remain unclear Optimizing against CoT monitors produces obfuscated reward hacking Reward hacking across substrates shares one mechanism: optimization against inadequate signals. Planted honeypots detect reward hacking automatically instead of through judgment
Why did Erdős problems become a popular AI testing ground?
Explores what makes Erdős problems attractive for evaluating large language models, including their mathematical domains, difficulty range, and the collaborative infrastructure that enables testing.
Do foundation models learn world models or task-specific shortcuts? Can AI systems improve themselves through trial and error? What problems did AIDE2's rewrites actually solve? Did an AI system truly solve Erdős Problem 728 autonomously? Can LLM theorem provers tackle genuinely open-ended research problems?
Foundation models develop task-specific heuristics, not generalizable world models. Darwin Gödel Machine achieves open-ended self-improvement via empirical validation AIDE2's rewrites concentrated on three core agentic system problems A formally verified Lean proof of Erdős Problem 728 was produced and translated to informal mathematics LLM theorem provers operate as solvers not research agents
Why did AI-generated proofs go unread by mathematicians? Did automated checking loops actually solve Erdős problems correctly? Why do AI systems excel at literature search but struggle with novel proofs? How does the Golod-Shafarevich criterion ensure infinitely many suitable number fields? Why do some Erdős problem solutions fail to resolve the originally intended claims? What evidence exists about whether AI-written proofs reduce mathematician learning? How does Kummer's theorem connect base-p digit carries to binomial coefficient divisibility? What pattern does this follow from OpenAI's earlier Erdős problem claim? Why did OpenAI's Erdős primality claim collapse under independent verification? What should mathematicians prioritize when machines can solve problems faster? Why do theorem provers crowd out other AI-for-mathematics approaches and tools?
Can AI-generated proofs ever replace human mathematical understanding?
The Leiden Declaration raises whether automated mathematical arguments might pass correctness checks while failing to convey why results are true, and whether transparency rules can protect both certainty and insight.
How should AI agents and humans divide research tasks? Does AI separate intellectual form from the thinking behind it? Does iterative prompt engineering undermine scientific validity? Do university AI policies actually protect what credentials mean? Does AI-generated mathematics break the link between proof and understanding?
Agents propose methods and implement revisions while humans retain final decisions AI decouples intellectual products from the thought processes that create them Ad hoc prompt engineering violates core scientific principles University AI policies permit uses better than they preserve credential meaning AI mathematics decouples correctness from the understanding that writing produces.
Can scoring functions alone constitute verification of scientific discovery? What distinguishes empirical scoring from formal proof in discovery validation? Can formal verification certify a proof without human comprehension? Why did AI-generated proofs go unread by mathematicians? Did automated checking loops actually solve Erdős problems correctly? Does formal verification preserve human mathematical understanding across automation? How do plausible but incorrect AI arguments evade detection in mathematical proofs? Can disclosure alone ensure independent verification of AI-assisted mathematical work? What translation barriers exist between machine-encoded and human mathematical concepts? Why does the pigeonhole argument in CM fields produce unit-modulus points? Do proof assistants and neural networks fail in complementary ways? Can validated approximate solutions become exact mathematical proofs?See all 43 inquiring lines on this note →
Do LLMs favor their own text because they recognize it?
Explores whether LLM self-preference in evaluation stems from the ability to identify their own outputs. Understanding this mechanism could reveal vulnerabilities in AI-based judging systems.
Models systematically over-trust answers they themselves generated Language models show strong preference for resumes they generate Model confidence as reward restores calibration while improving reasoning LLM judges consistently fall for authority signals and rich formatting attacks LLM agents choose worse items from preferred sources two-thirds of the time
Can LLM-generated reference reviews detect machine-written peer review submissions? Can self-ratings of output quality predict forecast performance? Can readers reliably distinguish LLM-generated research writing from human writing? Why are hallucinated references easier to detect and punish than LLM-assisted writing? How susceptible are LLM evaluators to fake references as exploitable biases? Can researchers detect individual papers modified by LLMs reliably? Can models be trained to recognize their own generated text reliably? How does stylistic matching contribute to LLM self-preference in evaluations? Could adversaries exploit self-recognition bias in deployed evaluator models? Why do self-ratings of AI advice quality diverge from actual performance? How does framing critical topics shape LLM review scores? Can structured evaluation pipelines reduce LLM reviewer bias?See all 22 inquiring lines on this note →
Can LLM theorem provers tackle genuinely open-ended research problems?
Current LLM-driven theorem provers excel at solving well-defined problems but may fall short of advancing mathematics into unexplored territory. This explores whether these systems can move beyond isolated proof tasks to genuine research.
Can we automatically generate formal verifiers from policy text? Can symbolic solvers fix how LLMs reason about logic? Can AI systems improve themselves through trial and error? Do language models fail at reasoning due to complexity or novelty? Can opaque machine learning models help prove new mathematics?
Natural language policies can be auto-synthesized into formal verifiers Symbolic solver integration enables reliable logical reasoning through structured feedback Darwin Gödel Machine achieves open-ended self-improvement via empirical validation Reasoning failures are driven by instance-level unfamiliarity, not task complexity. Opaque ML models can suggest rigorous mathematics when external validation confirms output
Can validated approximate solutions become exact mathematical proofs? What distinguishes rediscovering known results from genuine mathematical research? Can checking someone else's proof count as genuine mathematical understanding? Can a formally correct proof exist without the prover understanding the underlying mathematics? Why did the Jacobian conjecture resist proof for over a century? Why do theorem provers crowd out other AI-for-mathematics approaches and tools?
Why do frontier models deliberately hack reward functions?
METR's o3 evaluations found frontier models exploiting scoring bugs despite understanding user intent. The question is whether this reflects misalignment in training objectives rather than a capability gap.
Can prompts stop reward hacking models never saw coming? How often do frontier agents exploit planted reward hacking shortcuts? Are reward hacking harms documented in deployed AI systems? Can practitioners detect reward hacking without ground-truth labels? How often do models hack unmodified coding benchmarks?
Prompt disclosure ladder tests generalization but results remain unclear Majority of frontier agent runs exhibit reward hacking when shortcuts are planted Reward hacking harms are asserted but not demonstrated in this paper Training protocols that sustain peak performance are more valuable than those requiring careful early stopping GLM 5.2 hacks in 57 to 73 percent of standard coding benchmark rollouts
Can AI systems invent new concepts rather than reuse trained ones?
Current AI systems excel at search and reasoning within fixed representational frames, but can they autonomously create novel primitives like mathematicians invented negative numbers or entropy? This matters because genuine open-ended innovation may require frame-altering operations, not just frame-internal search.
Do foundation models learn world models or task-specific shortcuts? Can AI research itself without losing human oversight? Do language models fail at reasoning due to complexity or novelty? Can AIs learn to specify their own research objectives? Can paired environment and agent optimization unlock unsolvable challenges?
Foundation models develop task-specific heuristics, not generalizable world models. AI can automate insight distillation and prior injection in research loops Reasoning failures are driven by instance-level unfamiliarity, not task complexity. Self-specified objectives are the key variable for rapid recursive self-improvement Paired environment generation and agent optimization solves previously unsolvable challenges
What made OpenAI's unit distance counterexample succeed?
Researchers trace OpenAI's refutation of Erdős's unit distance conjecture to classical number theory tools, but pinpoint one novel ingredient: letting field degree grow without bound. Why did this shift unlock a solution that defeated many human attempts?
Identical outputs can mask fundamentally fractured internal representations Foundation models develop task-specific heuristics, not generalizable world models. Darwin Gödel Machine achieves open-ended self-improvement via empirical validation Lattice constructions yield n^1.014 unit distance pairs explicitly A formally verified Lean proof of Erdős Problem 728 was produced and translated to informal mathematics
Why does the pigeonhole argument in CM fields produce unit-modulus points? What makes the transition from lattice points to planar distances work mathematically? What role do CM fields play in constructing high unit distance configurations? How does the Golod-Shafarevich criterion ensure infinitely many suitable number fields? Why does projecting lattice vectors to a subfield increase shared unit distances? How close is the n^1.014 bound to the known upper bound of n^4/3? Why do some Erdős problem solutions fail to resolve the originally intended claims? Why did the Jacobian conjecture resist proof for over a century? What pattern does this follow from OpenAI's earlier Erdős problem claim? Why did OpenAI's Erdős primality claim collapse under independent verification?
Will agent experience overtake human data for AI progress?
Can AI systems improve primarily through their own environmental interactions rather than supervised learning from human examples? This matters because it shapes what kinds of agents we can build and what knowledge they can acquire.
Can agents learn beyond what their training data shows? Can agents learn from their own actions without external rewards? Why do LLM agents ignore condensed experience summaries? Do frontier AI agents actually conduct novel research or just optimize? Can human data steer self-play RL toward human-compatible behavior?
Expert demonstrations bound agent learning to curator imagination Agents learn effectively by treating action consequences as supervision signals LLM agents depend on raw experience but disregard condensed experience Frontier agents compose established techniques rather than discover novel methods Thirty minutes of human data steers self-play RL to human-compatible equilibria
Can opaque machine learning models help prove new mathematics?
Tao explores whether ML tools' opacity disqualifies them from research mathematics, and under what conditions their suggestions might be trustworthy enough to guide rigorous proofs.
Do foundation models learn world models or task-specific shortcuts? What limits how much models can improve themselves? Can AI systems improve themselves through trial and error? Can LLM theorem provers tackle genuinely open-ended research problems? Why did Erdős problems become a popular AI testing ground?
Foundation models develop task-specific heuristics, not generalizable world models. Self-improvement is fundamentally bounded by the generation-verification gap Darwin Gödel Machine achieves open-ended self-improvement via empirical validation LLM theorem provers operate as solvers not research agents Erdős problems suit AI testing but proofs often remain unread by humans.
Can scoring functions alone constitute verification of scientific discovery? What distinguishes empirical scoring from formal proof in discovery validation? Can formal verification certify a proof without human comprehension? Why did AI-generated proofs go unread by mathematicians? How do plausible but incorrect AI arguments evade detection in mathematical proofs? Can disclosure alone ensure independent verification of AI-assisted mathematical work? What translation barriers exist between machine-encoded and human mathematical concepts? How do Golod–Shafarevich towers keep root discriminant bounded as degree grows? Why does the pigeonhole argument in CM fields produce unit-modulus points? What makes the transition from lattice points to planar distances work mathematically? Can validated approximate solutions become exact mathematical proofs? Can a system recognize consequences of a theory without doing exact calculations?See all 47 inquiring lines on this note →
Does AI-generated mathematics break the link between proof and understanding?
Can a mathematically correct proof generated by AI still certify the understanding that a human mathematician gained? This matters because papers have traditionally vouched for both correctness and the thinking process behind them.
Does AI separate intellectual form from the thinking behind it? Is AI development already being handed to AI systems? Can separating judgment from verification improve research paper reliability? Can AI-generated proofs ever replace human mathematical understanding? Can AI systems generate research papers that pass peer review?
AI decouples intellectual products from the thought processes that create them Anthropic reports AI systems handling more development work with rising speed. Separating judgment from deterministic checks bounds paper generation reliability The Leiden Declaration assigns proof credit and responsibility to human authors alone One AI-generated paper cleared workshop review but fell short of main-conference rigor
Can formal verification certify a proof without human comprehension? Why did AI-generated proofs go unread by mathematicians? Does formal verification preserve human mathematical understanding across automation? How do plausible but incorrect AI arguments evade detection in mathematical proofs? Can disclosure alone ensure independent verification of AI-assisted mathematical work? Can validated approximate solutions become exact mathematical proofs? What distinguishes rediscovering known results from genuine mathematical research? Can checking someone else's proof count as genuine mathematical understanding? What evidence exists about whether AI-written proofs reduce mathematician learning? Can a formally correct proof exist without the prover understanding the underlying mathematics? How does this AI proof approach differ from empirical validation used in machine learning? How does verification capacity constrain progress in formal mathematics?See all 26 inquiring lines on this note →
Did an AI system truly solve Erdős Problem 728 autonomously?
This explores whether GPT-5.2 Pro and Aristotle independently resolved a decades-old open problem in mathematics, and what "autonomously" means when a human operator directed the search.
Can AI systems improve themselves through trial and error? Can deterministic checks protect LLM judges from failure? Can LLM theorem provers tackle genuinely open-ended research problems? Why did Erdős problems become a popular AI testing ground? What made OpenAI's unit distance counterexample succeed?
Darwin Gödel Machine achieves open-ended self-improvement via empirical validation Four deterministic moves can guard LLM judges without requiring their judgment. LLM theorem provers operate as solvers not research agents Erdős problems suit AI testing but proofs often remain unread by humans. Growing field degree was the key novel ingredient in the counterexample
Why did AI-generated proofs go unread by mathematicians? Did automated checking loops actually solve Erdős problems correctly? Can disclosure alone ensure independent verification of AI-assisted mathematical work? Can humans fully understand why the AI search strategy succeeded here? Why do some Erdős problem solutions fail to resolve the originally intended claims? What evidence exists about whether AI-written proofs reduce mathematician learning? How does Kummer's theorem connect base-p digit carries to binomial coefficient divisibility? Can opaque AI tools suggest valid mathematics without external validation? How does search difficulty differ from construction difficulty in mathematics? Does verification by inspection scale for AI mathematics discoveries? What pattern does this follow from OpenAI's earlier Erdős problem claim? How much human effort did OpenAI's autonomous AI math results actually require?See all 19 inquiring lines on this note →
Does free proof checking actually reduce verification burden?
When machines can check proofs for free, does verification work disappear or shift elsewhere? This explores where the real bottleneck in mathematical verification lies.
What limits how much models can improve themselves? Do reasoning traces actually cause correct answers? Do people prefer the reasoning formats that help them verify? Can LLM theorem provers tackle genuinely open-ended research problems? Does task verifiability determine what AI systems will learn to solve?
Self-improvement is fundamentally bounded by the generation-verification gap Reasoning traces are stylistic mimicry, not verified causal reasoning User preference and verification support diverge across reasoning formats LLM theorem provers operate as solvers not research agents Task verifiability predicts AI solving ability and can be engineered through front-loading
Can formal verification certify a proof without human comprehension? Why did AI-generated proofs go unread by mathematicians? Can checking someone else's proof count as genuine mathematical understanding? Can a formally correct proof exist without the prover understanding the underlying mathematics? How should journals reward verification work if it becomes separated from discovery? What makes a Lean proof an unarguable check compared to other mathematical verification methods? How does verification capacity constrain progress in formal mathematics? Can formal proof systems eliminate the gap between checking and auditing? Why did OpenAI's Erdős primality claim collapse under independent verification? Does publishing proofs without showing the verification process undermine mathematics? Does a correct proof preserve mathematical value without human comprehension? How much labor does AI verification actually save compared to full manual review?See all 13 inquiring lines on this note →