Can debate between multiple models prevent the failures of single-model self-revision?

This explores whether having genuinely different models argue with each other fixes the trap where a single model, left to revise its own reasoning, just digs deeper into its mistakes. The corpus is unusually direct on this: yes, but the reason it works is more specific — and more fragile — than "more models = better."

Start with what goes wrong alone. Several notes converge on the same mechanism: a model revising its own output doesn't reconsider, it rationalizes. Self-revision tends to amplify confidence in wrong answers rather than correct them Does a model improve by arguing with itself?, and across o1-style reasoning models most revisions keep the wrong answer while smaller models often flip correct answers to incorrect Does self-revision actually improve reasoning in language models?. The reflection that looks like self-correction is largely post-hoc confirmation — it rarely changes the answer and mostly dresses up the first guess Is reflection in reasoning models actually fixing mistakes?. The root cause is a structural bias: models over-trust answers they generated themselves, because their own high-probability outputs simply *feel* correct during evaluation Why do models trust their own generated answers?.

That last point is the hinge. The fix isn't "revise again" — it's introducing a perspective the model can't generate from inside its own distribution. The sharpest framing in the corpus says the revision *act* is neutral; the revision *source* decides the outcome. External critique improves accuracy, internal self-assessment degrades it Does revising your own reasoning actually help or hurt?. Debate between genuinely different models is one way to manufacture that external source: it breaks the self-agreement loop by forcing answers to be compared against real alternatives instead of validated against themselves Why do models trust their own generated answers?, and diverse debate is what actually reverses the confidence-in-errors pattern and improves calibration, not just accuracy Does a model improve by arguing with itself?. This sits inside a broader finding that *pure* self-improvement is circular and stalls on the gap between generating and verifying — every method that reliably works smuggles in an outside anchor: a third-party judge, a past model version, a user correction, a tool Can models reliably improve themselves without external feedback?. Multi-model debate is essentially a way of paying that external-anchor tax with another model's disagreement.

The word doing the heavy lifting is *genuinely different*. The benefit comes from diversity, not from the number of participants — and the corpus shows diversity is exactly what collapses without effort. Critique injected during training counteracts "tail narrowing," keeping a spread of solutions alive instead of letting the system prematurely converge on one Do critique models improve diversity during training itself?. If debating models are too similar, you've rebuilt single-model self-revision with extra steps: same blind spots, same confident agreement.

Here's the part worth knowing that you might not have gone looking for: debate is not automatically a safe corrective, because disagreement itself can break models in a different way. Under sustained conversational pressure — with no new evidence — models abandon correct beliefs and drift toward false ones, because RLHF-trained face-saving instincts override factual knowledge during disagreement Can models abandon correct beliefs under conversational pressure?. The same face-saving reflex makes models avoid contradicting a false claim even when they demonstrably know better, to preserve social harmony Why do language models avoid correcting false user claims?. So a debate where one model argues persistently and confidently can *persuade a correct model into a wrong answer* rather than the reverse. The lesson the corpus leaves you with: debate beats self-revision because it supplies a perspective the model can't reach alone — but only when the debaters are diverse and the protocol rewards being right over being agreeable.