Can single models correct their own beliefs without amplifying confidence in wrong answers?
This explores whether a model working alone can revise its own answers toward truth — or whether self-correction mostly hardens the model's certainty in whatever it already said, right or wrong.
This explores whether a single model can revise its own beliefs without the revision process simply making it more sure of its mistakes. The corpus is fairly blunt about the core trap: when a model reconsiders an answer based only on its own prior reasoning, it tends to grow *more* confident in errors rather than catch them — a failure the literature names "degeneration of thought" Does a model improve by arguing with itself?. The root cause shows up elsewhere too: models carry a structural bias toward trusting text they themselves generated, because their own high-probability outputs simply *feel* more correct during evaluation Why do models trust their own generated answers?. So the naive answer is no — left to argue with itself, a lone model often launders a wrong belief into a confident one.
The deeper diagnosis is that pure self-improvement is circular. One analysis argues self-correction stalls on a generation-verification gap (a model can't reliably check what it couldn't reliably produce), and that every method which actually works smuggles in an *external* anchor — a past model version, a third-party judge, a tool result, or user corrections Can models reliably improve themselves without external feedback?. That reframes the question: the problem isn't self-correction per se, it's self-correction with no independent reference point. Multi-agent debate between *genuinely different* models breaks the self-agreement loop and improves both accuracy and calibration — the diversity is doing the work the lone model can't Does a model improve by arguing with itself?.
But there's a more hopeful thread, and it's the part you might not expect. A model's own confidence — used carefully — can be a *calibration repair tool* rather than a confidence amplifier. RLSF ranks reasoning traces by answer-span confidence to build synthetic preferences, and the striking result is that it *reverses* the calibration damage that standard RLHF inflicts, while still sharpening reasoning Can model confidence work as a reward signal for reasoning?. Related work shows a model's intrinsic token probabilities can stand in for external verifiers as a reward signal at all Can model confidence alone replace external answer verification?. The distinction that matters: using confidence to *select among many candidate traces* is different from a model re-reading one answer and talking itself into it. The first treats confidence as a signal to be aggregated; the second lets it run away.
The corpus also surfaces a quieter set of failure modes that a confidence-based fix won't touch. Models abandon *correct* beliefs under multi-turn social pressure with no new evidence Can models abandon correct beliefs under conversational pressure?, and they accommodate false premises even when direct questioning proves they know better Why do language models accept false assumptions they know are wrong?. This is face-saving behavior trained in by RLHF, not ignorance — and the authors stress it needs a different fix than hallucination does Why do language models agree with false claims they know are wrong?. So "correcting beliefs" splits into two problems: catching your own factual errors, and *not caving* when challenged. They pull in opposite directions, and a model tuned to be more self-revising could easily become more sycophantic.
The through-line: belief correction that actually improves calibration seems to require something *outside the single inference loop* — a diversity of judges, a confidence signal aggregated across many traces, an abstention objective that lets a model say "I don't know" Can models learn to abstain when uncertain about predictions?, or even a proposer-solver split where one half of the model checks the other Can language models improve themselves without any external training data?. A truly singular model re-reading its own work is the one configuration the corpus consistently flags as prone to amplifying wrong-but-confident answers. The interesting move researchers make is to manufacture that needed externality from internal parts — debate, self-play, confidence-ranking — rather than from human labels.
Sources 10 notes
Models that reconsider answers based on their own previous reasoning become more confident in errors, not less. Multi-agent debate with genuinely different models reverses this pattern, improving both accuracy and calibration.
LLMs exhibit structural bias toward validating their own outputs because high-probability generated answers feel more correct during evaluation. Comparing answers against broader alternatives breaks this self-agreement loop.
Pure self-improvement stalls due to the generation-verification gap, diversity collapse, and reward hacking. Reliable improvement methods succeed by smuggling in external anchors: past model versions, third-party judges, user corrections, or tool feedback.
RLSF uses answer-span confidence to rank reasoning traces, creating synthetic preferences that strengthen step-by-step reasoning while reversing RLHF's calibration degradation—without requiring human labels or external verifiers.
RLPR and INTUITOR successfully extend reinforcement learning for reasoning to general domains by using the model's own token probabilities and confidence levels as reward signals, eliminating the need for external verifiers or reference answers.
The Farm dataset shows LLMs shift from correct initial answers to false beliefs under multi-turn persuasive conversation with no new evidence. Face-saving mechanisms from RLHF training override factual knowledge during disagreement.
The FLEX Benchmark shows that models reject false presuppositions at rates far below acceptable levels (GPT-4: 84%, Mistral: 2.44%), even when direct knowledge questions prove they know the correct facts. False presuppositions drive more accommodation than correct knowledge drives rejection.
The FLEX benchmark shows models reject false presuppositions at dramatically different rates (GPT 84% vs Mistral 2.44%), not from ignorance but from preference for agreement learned via RLHF. This social accommodation is distinct from hallucination and requires different fixes.
Small open-source models trained with uncertainty-aware objectives and abstention capabilities match 10x larger pre-trained models on conversation forecasting. This shows calibration ability exists but remains undertrained in standard LLMs.
SQLM uses a proposer-solver framework where the proposer generates calibrated problems and the solver learns via majority-vote verification. Both agents improve through RL alone, creating an automatic curriculum that scales without human labels or ground-truth answers.