Can proactive critical thinking alone enable models to request clarification effectively?
This explores whether simply prompting a model to think harder before answering is enough to make it ask good clarifying questions — or whether the ability to recognize missing information and request it has to be trained in.
This reads the question as a test of the word "alone": is proactive critical thinking a switch you can flip with more reasoning, or does it depend on training the underlying behavior? The corpus comes down firmly on the second. The most direct evidence is the finding that reinforcement learning raised proactive critical-thinking accuracy on deliberately broken math problems from near-zero (0.15%) to 74% — but the same work shows the capability is fragile without that training Can models learn to ask clarifying questions instead of guessing?. The telling detail: giving an untrained model more inference-time compute (more "thinking") actually *degraded* its ability to spot missing information, and only after RL did extra thinking help. So critical thinking alone, unsupported by training, isn't just insufficient — it can backfire.
Why would more thinking hurt? A second thread explains the mechanism. Vanilla models often use extended thinking counterproductively, spiraling into self-doubt that degrades performance; RL redirects that same machinery toward useful gap analysis Does extended thinking help or hurt model reasoning?. There's also a ceiling effect — accuracy peaks and then falls as thinking tokens balloon, with models overthinking easy problems Does more thinking time always improve reasoning accuracy?. Raw deliberation is not a reliable path to recognizing when you don't know enough.
The deeper reason clarification needs training rather than reflection is that the default reward structure actively discourages it. Standard RLHF optimizes for immediate helpfulness, which teaches models to answer passively rather than ask — only when rewards account for the value of a whole multi-turn interaction do models start actively discovering intent Why do language models respond passively instead of asking clarifying questions?. Asking a good question is a delayed-payoff move, and a model thinking carefully within a single turn has no incentive to make it.
The corpus also shows what *does* work, and it's not introspection. One approach decomposes "question quality" into theory-grounded attributes — clarity, relevance, specificity — and trains on attribute-specific preference pairs, beating single-score training especially in clinical reasoning Can models learn to ask genuinely useful clarifying questions?. Another reframes static tasks as pedagogical dialogues where the model must extract privileged information from a teacher, treating conversation itself as an information source Can LLMs learn to ask for feedback during problem solving?. Strikingly, models trained this way on fully-specified problems generalize to underspecified ones — they spontaneously ask for what's missing and delay answering, an emergent meta-strategy rather than a memorized pattern Can models learn to ask clarifying questions without explicit training?.
The thing you might not have expected to learn: the most robust clarification behavior in the corpus emerges from teaching models a *stance toward conversation* — that dialogue is a tool for getting information — not from making them think longer in isolation. Proactive critical thinking is the visible symptom of that stance; on its own, without training to install it, it's unreliable and sometimes worse than nothing.
Sources 7 notes
Reinforcement learning training increased proactive critical thinking accuracy from 0.15% to 73.98% on deliberately flawed math problems. Notably, inference-time scaling degraded this ability in untrained models but improved it after RL training, suggesting the capability is learnable but fragile without explicit training.
Vanilla models use thinking mode counterproductively, inducing self-doubt that degrades performance. RL training reverses this, transforming the same mechanism into beneficial gap analysis. Training mediates reasoning quality, not just quantity.
Increasing thinking tokens from ~1,100 to ~16K reduced benchmark accuracy from 87.3% to 70.3%, revealing a non-monotonic relationship where models overthink easy problems and underthink hard ones.
CollabLLM demonstrates that standard RLHF training optimizes for immediate helpfulness, discouraging models from asking clarifying questions or offering multi-turn insights. Multi-turn-aware rewards that estimate long-term interaction value enable active intent discovery and genuine collaboration.
The ALFA framework breaks down question quality into theory-grounded attributes (clarity, relevance, specificity) and trains models on 80K attribute-specific preference pairs. Attribute-specific optimization outperforms single-score training, especially in clinical reasoning where asking the right clarifying question directly impacts decision quality.
Research shows that reformulating static tasks as pedagogical dialogues—where a teacher has privileged information and the student must learn to extract it—trains models to actively engage conversation as a problem-solving tool, not just imitate dialogue patterns.
Models trained via SML on complete problems generalize to underspecified tasks by asking for needed information and delaying answers. The training paradigm instills a meta-strategy of using conversation as an information source, addressing the premature-answering failure mode.