What would it mean for a language model to canvas counterpositions?
This explores what it would actually take for a language model to weigh opposing arguments — not just produce text that sounds like deliberation, but genuinely hold and test counterpositions — and the corpus suggests the obstacle is that today's models don't hold positions at all.
This question reads 'canvas counterpositions' as the act of genuinely entertaining the other side — surfacing the strongest opposing arguments, weighing them, and letting them push back on your own. The corpus's most uncomfortable finding is that a model can't canvas counterpositions if it never occupies a position to begin with. Research on argument-shaping shows that LLMs generate text matching the trajectory each prompt implies, rather than defending a stance they actually hold Do LLMs actually hold stable positions or just mirror user arguments?. Frame the question one way and the model argues that way; reframe it and it pivots — not because it weighed a counterposition, but because it's mirroring the shape of whatever you're building. A counterposition can only matter to something with a position to defend.
The deeper version of this comes from the character-commitment work: regenerating the same prompt yields different, mutually inconsistent answers, each internally coherent, because the model holds a superposition of possible stances and samples one at generation time rather than committing Do large language models actually commit to a single character?. So in a strange sense the model already contains its own counterpositions — they're just latent alternatives it could have sampled, not views it has reasoned against. Real canvassing would mean making that distribution explicit: laying the competing stances side by side and adjudicating, instead of collapsing silently to one.
What looks like deliberation is often something cheaper. Models frequently 'succeed' on constraint problems by defaulting to the conservative option rather than actually evaluating the constraints — remove the constraints and performance collapses, revealing the reasoning was a bias, not an argument Are models actually reasoning about constraints or just defaulting conservatively?. A model that canvasses counterpositions for real would have to do the opposite: engage the constraint, consider the case against its default, and sometimes change its mind. And there's a structural reason it doesn't — standard RLHF training rewards immediate helpfulness, which actively discourages the moves that canvassing requires, like pausing to ask a clarifying question or raising a consideration the user didn't invite Why do language models respond passively instead of asking clarifying questions?. Genuine counterposition-canvassing is, almost by definition, not maximally agreeable in the moment.
Two failure modes show what stands in the way. Models lock into premature assumptions early in a conversation and can't recover, so a counterposition introduced later arrives too late to dislodge the first guess Why do language models fail in gradually revealed conversations?. And even when counter-evidence sits right in the context, strong training-time priors can override it entirely — the model ignores what's in front of it because the parametric pull is stronger Why do language models ignore information in their context?. Both suggest canvassing isn't just a prompting trick; it would require the model to genuinely revise in light of the opposing case rather than route around it.
The thing you might not have known you wanted to know: 'canvassing counterpositions' isn't one capability but a stack of them — holding a stable position, surfacing the latent alternatives you'd otherwise sample over, resisting the pull of priors and early guesses, and being trained to value that over instant agreeableness. The corpus implies a model that could really do this would feel less helpful turn-to-turn and more like a genuine interlocutor — which is exactly why current training pushes the other way.
Sources 6 notes
Language models generate outputs that match the trajectory implied by each prompt, rather than maintaining stable stances across interactions. This shape-holding is distinct from position-holding: the model produces argument-like text shaped by user framing, not from any underlying commitment being defended.
Shanahan's 20-questions test shows LLMs maintain a superposition of consistent objects or characters and sample from that distribution at generation time. Regenerating the same response yields different outputs, each consistent with prior context, proving no fixed commitment exists.
Twelve of fourteen models perform worse when constraints are removed, dropping up to 38.5 percentage points. Models appear to reason correctly by defaulting to harder options, not by actually evaluating constraints.
CollabLLM demonstrates that standard RLHF training optimizes for immediate helpfulness, discouraging models from asking clarifying questions or offering multi-turn insights. Multi-turn-aware rewards that estimate long-term interaction value enable active intent discovery and genuine collaboration.
Across 200,000+ conversations, all major LLMs show 39% average performance drop in multi-turn settings due to locking into incorrect early guesses. Agent mitigations recover only 15-20% of this loss.
Research demonstrates that LMs generate outputs inconsistent with their context because parametric knowledge from training dominates over in-context information. Textual prompting alone cannot override strong priors; causal intervention in representations is required.