When you give an AI a fuzzy request, why does it guess confidently instead of asking what you meant?
How do underspecified goals reveal gaps in AI assistance?
This explores what happens when a request is vague or incomplete, and what the ways assistants stumble on it show about what they lack.
This explores what happens when a request is vague or incomplete, and what an assistant's stumbles reveal about what it's missing. The corpus points to three gaps: assistants don't track what they don't know, they don't ask, and they don't do the work of turning a fuzzy goal into something checkable.
The first gap is that assistants have no place to put their ignorance. Research finds that language models carry Do language models know what they don't know about users?: nothing in the model marks which facts about the person are missing. So a vague request gets filled with a confident guess, which shows up as sycophancy or hallucination. When researchers added a schema of labeled unknowns to the prompt, harmful advice and sycophancy dropped by 50–75% and hallucination roughly halved. The vagueness was never the whole problem. The model simply had no way to say 'I don't know this yet.'
The second gap is behavioral. Even if a model could notice what's missing, current assistants are built to respond, not to lead. Why can't conversational AI agents take the initiative? argues that training rewards answering queries rather than pursuing goals, so they rarely stop to probe. The problem gets worse with tools. When should AI agents ask users instead of just searching? describes how agents drift from what the user wanted by chaining tool calls silently. It borrows a fix from conversation analysis: the short clarifying question people slot in mid-conversation ('do you mean the old version or the new one?'). That prevents the misunderstanding rather than repairing it afterward.
The third gap is that turning a vague goal into a workable one is real work that most AI systems skip. Can agents learn from vague goals without predefined metrics? gives agents only a natural-language direction, with no tasks or metrics. They first have to build their own training and validation signals before they can improve at all. Most methods assume that step is already done. A related move appears in Can breaking down instructions into checklists improve AI reward signals?, where a subjective instruction is broken into verifiable sub-criteria so a system can tell whether it succeeded. Both approaches are about making a fuzzy goal checkable before optimizing it.
There's also a deeper worry. Can AI systems achieve real alignment without world contact? argues that a goal encoded purely as symbols, with no contact with the world or the people behind it, can drift from what was meant. An underspecified request is that gap in miniature: the words the user typed and what they wanted are different things. Closing it seems to take three things: representing what's unknown, being willing to ask about it, and treating goal interpretation as its own step of the work.
Sources 6 notes
Research shows assistants suffer from sycophancy and hallucination because they have no representation of what remains unknown about users. Adding a schema of labeled unknowns to prompts reduced harmful advice and sycophancy by 50–75% and cut hallucination rates by roughly half.
Research shows LLMs including ChatGPT cannot initiate topics, plan strategically, or lead conversations because their training optimizes for responding to queries, not creating dialogue from agent goals. This passivity is reinforced by alignment objectives and masked by fluent-sounding outputs.
Tool-enabled LLMs drift from user intent through silent tool chaining. Conversation analysis reveals insert-expansions—clarifying intent, scoping responses, enhancing appeal—as a formal framework for proactive user consultation that prevents misunderstanding instead of recovering from it.
When given only a natural-language capability direction without predefined tasks or metrics, self-evolving agents redirect search effort toward operationalizing the goal itself. Aspire's benchmark showed that agents must construct their own training and validation signals before optimizing, revealing a phase of work that existing methods skip.
RLCF and RaR methods decompose instruction quality into verifiable sub-criteria, improving performance on benchmarks like FollowBench and HealthBench. This decomposition principle reduces overfitting to superficial artifacts that plague holistic reward models.
Show all 6 sources
Peircean semiotics reveals that symbolic goal encoding without world contact and social mediation cannot guarantee correspondence to actual values. LLMs operating in pure symbol manipulation risk divergence between stated goals and real-world outcomes.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Proactive Conversational Agents in the Post-ChatGPT World
- DiscussLLM: Teaching Large Language Models When to Speak
- Intent Mismatch Causes LLMs to Get Lost in Multi-Turn Conversation
- Conversational Alignment with Artificial Intelligence in Context
- RLVMR: Reinforcement Learning with Verifiable Meta-Reasoning Rewards for Robust Long-Horizon Agents
- Beyond Hallucinations: The Illusion of Understanding in Large Language Models
- Aspire: Can Models Self-Evolve from Vague Goals?
- Checklists Are Better Than Reward Models For Aligning Language Models