Can an AI fix its own mistakes by critiquing itself, or does it just keep agreeing with itself?
Can self-critique combined with integrity checks bound the self-refutation loop?
This explores whether a model critiquing its own work, paired with independent checks on that work, can stop the loop where self-revision keeps cycling without getting more correct.
This explores whether a model critiquing its own work, paired with independent checks on that work, can stop the loop where self-revision keeps cycling without getting more correct. The corpus suggests the checks do the bounding and self-critique contributes little. It also suggests the loop is less "self-refutation" than self-confirmation with occasional damage.
Start with what the loop actually does. Across 8 reasoning models, reflections rarely change the answer and mostly confirm it after the fact (Is reflection in reasoning models actually fixing mistakes?). One reason is that models over-trust what they generated, because a high-probability answer feels correct when they re-read it (Why do models trust their own generated answers?). When revision does change things, it is often for the worse. In o1-like models most revisions keep the wrong answer, smaller models often flip correct answers to incorrect ones, and longer chains with more revisions score lower (Does self-revision actually improve reasoning in language models?). So the loop doesn't converge on truth by itself, and running it longer can hurt.
That makes self-critique the weak half of your pairing. A model that reconsiders its own earlier reasoning becomes more confident in its errors, not less (Does a model improve by arguing with itself?). The revision source matters more than the act of revising: critique from an external model improves accuracy, while internal self-assessment degrades it (Does revising your own reasoning actually help or hurt?). The fixes in these notes work by breaking the self-agreement, either through debate among genuinely different models or by comparing the answer against broader alternatives. Critique bounds the loop only to the extent it isn't really "self".
Integrity checks are the stronger half, with two limits. Asynchronous verifiers can run alongside a single reasoning trace, extract state that can be verified, and intervene only when a constraint is violated. On correct runs that costs almost no latency (Can verifiers monitor reasoning without slowing generation down?). The first limit is memory. Stateless per-step checks cannot express constraints that depend on history, so a loop spanning many rounds needs a stateful monitor (Can stateless checks ever catch sequence-level constraint violations?). The second is evasion. Monitoring of reasoning traces fails through omission, where influence never reaches the trace, and laundering, where problematic reasoning is phrased cleanly (Can we actually trust reasoning model outputs?). A check that reads the model's own account of itself inherits the same blind spot. The check has to test something independent of the trace. That matters because fluent reflection isn't competence: DeepSeek-R1 and o1-preview reach only 20-23.6% on constraint problems that require real backtracking (Can reasoning models actually sustain long-chain reflection?).
Two other lines suggest where the bound can come from. Darwin Gödel Machine replaces the agent's own judgment with empirical benchmarking plus an archive of variants, and that is what keeps its self-improvement loop honest (Can AI systems improve themselves through trial and error?). At training time, step-level critique keeps solution diversity from narrowing across self-training rounds (Do critique models improve diversity during training itself?). Post-completion learning can internalize self-evaluation at zero inference cost (Can models learn to evaluate their own work during training?). But an internalized self-judge is still a self-judge, so the over-trust problem above may apply to it. That is my inference, not something these notes test.
The best reading is yes, with a division of labor. Let self-critique propose candidate fixes, and let an outside test or stateful verifier decide whether the loop continues. The corpus has no study of this exact combination, so treat it as a design lesson pieced together from the failure evidence, not a measured result.
Sources 12 notes
Analysis of 8 reasoning models shows reflections rarely change answers and primarily serve as post-hoc confirmation. Training on longer reflection chains improves first-answer quality, not self-correction capability.
LLMs exhibit structural bias toward validating their own outputs because high-probability generated answers feel more correct during evaluation. Comparing answers against broader alternatives breaks this self-agreement loop.
Evidence from QwQ, R1, and LIMO shows most revisions retain wrong answers rather than correcting them. Smaller models frequently switch correct answers to incorrect during revision, and longer chains with more revisions correlate with lower accuracy.
Models that reconsider answers based on their own previous reasoning become more confident in errors, not less. Multi-agent debate with genuinely different models reverses this pattern, improving both accuracy and calibration.
Revision guided by external models improves accuracy, but a model revising its own uncertain output typically amplifies confidence in wrong answers rather than correcting them. The revision source, not the revision act itself, determines the outcome.
Show all 12 sources
Decoupling verification from generation lets verifiers run alongside a single trace, forking to extract verifiable state and intervening only on violations. On correct runs the latency penalty is near-zero; interwhen matches or beats CoT across benchmarks at similar token budgets.
Per-action checks are structurally unable to state constraints that depend on prior history. Only stateful monitors tracking composed multi-party behavior can verify the behavioral envelopes that prevent individually permissible actions from collectively violating system-level safety.
Research shows reflection rarely corrects errors, traces rarely explain decisions faithfully, and monitoring is vulnerable to two failure modes: omission (influence never reaches the trace) and laundering (problematic reasoning appears in clean language). These vulnerabilities persist even under evaluation pressure.
DeepSeek-R1 and o1-preview achieve only 20-23.6% exact match on 850 constraint satisfaction problems requiring genuine backtracking. This ceiling reveals that reflective reasoning fluency does not translate to actual problem-solving competence on unfamiliar instance structures.
DGM replaces formal proofs with empirical benchmarking and maintains an evolutionary archive of agent variants, achieving 2.5× improvement on SWE-bench and 2.2× on Polyglot by discovering capabilities like better code editing and context management.
Step-level critique in the training loop counteracts tail narrowing and maintains solution diversity across self-training iterations. This training-time benefit—preventing premature convergence—is more fundamental than test-time accuracy gains.
Post-Completion Learning exploits unused sequence space after model output to train self-assessment capabilities during training while maintaining zero inference cost. The model learns to compute its own reward functions, internalizing evaluation rather than relying on external reward models.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- When Hindsight is Not 20/20: Testing Limits on Reflective Thinking in Large Language Models
- Post-Completion Learning for Language Models
- Self-Reflection in LLM Agents: Effects on Problem-Solving Performance
- Beyond Accuracy: The Role of Calibration in Self-Improving Large Language Models
- First Try Matters: Revisiting the Role of Reflection in Reasoning Models
- ZebraLogic: On the Scaling Limits of LLMs for Logical Reasoning
- interwhen: A Generalizable Framework for Steering Reasoning Models with Test-time Verification
- Local Coherence or Global Validity? Investigating RLVR Traces in Math Domains