Why do AI models struggle with unspoken user needs?
Can frontier models infer what users actually need when requests are casual and underspecified? This matters because real user requests often hide their true requirements beneath surface-level instructions.
xDailyBench is a benchmark of 248 curated tasks across 51 scenarios in personal life, white-collar work, learning and research, and cross-domain activities. The tasks are "grounded in requests that users have actually completed or genuinely intended to accomplish with AI," and they are scored with fine-grained binary rubrics that cover both explicit and implicit requirements. Across 11 frontier models run under standardized agentic settings, the best models reach a task-level score of 75.6%. Every model does "substantially worse on implicit than explicit requirements," with gaps "no less than 9 percentage points." The authors read this as implicit requirement inference being "a persistent bottleneck" for satisfying everyday user needs.
The paper's framing of why is about the shape of real requests. They are "open-ended, casually specified, and context-dependent," so a model has to do more than follow instructions. It must "infer unstated needs from user background and situational context." The introduction also argues that agent benchmarks have converged on autonomous end-to-end professional workflows, such as software engineering, web interaction and computer use, and that this is "only one practical role of AI agents." The discussion adds a process-level view. Tasks require gathering facts from several sources, reconciling constraints and producing an actionable deliverable. The authors name two failure modes: "missing feasibility checks" and "non-convergent evidence collection." In the first, a model produces a "seemingly complete" detailed plan without verifying that it is feasible, because users "typically ask directly for a detailed plan" and do not demand proof of feasibility. In the second, a model keeps collecting evidence and never converges on a synthesis and final artifact.
Against the neighbors, this paper adds a within-task axis. Does a single benchmark score actually predict agent readiness? argues that agent readiness decomposes into axes such as task success, privacy and retention. xDailyBench suggests the split also runs inside one task, between what the user said and what the user needed, and that an aggregate score can hide it. It also shifts the target relative to Why do AI agents fail at workplace social interaction?. That note measures autonomous completion in a simulated workplace, while xDailyBench measures consultation-style requests where the difficulty is under-specification. The two scores are not comparable, since the metrics and task sets differ, but both point to the gap between what users ask and what they need as a place where agents fail.
The excerpt does not report the per-model scores, how many rubric items are implicit versus explicit, or how the rubrics were written and validated. It also does not say which models did best or how the two failure modes are distributed across models. It does not say whether a missing feasibility check is itself scored as an implicit requirement. It also does not test any remedy for the gap. What follows at this strength is narrow. On this benchmark, a single headline score overstates how well models serve underspecified requests, and the implicit-versus-explicit gap is worth reporting as its own number.
Related concepts in this collection 2
This note in its neighbourhood — explore the map, then jump to a related concept in the list below.
Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph
-
Does a single benchmark score actually predict agent readiness?
Single-axis benchmarks rank models by one capability—like task success—but ignore privacy, duration, operating mode, and ecosystem fit. Can one number really capture what matters for deployment?
extends the separable-axes claim with an explicit-versus-implicit split inside a single task's score
-
Why do AI agents fail at workplace social interaction?
Explores why current AI agents struggle most with communicating and coordinating with colleagues in realistic workplace settings, despite strong reasoning capabilities in other domains.
contrasts autonomous professional-workflow completion with everyday consultation, where under-specified requests are the hard part
Related papers in this collection 8
Papers most semantically related to this note, ranked by cosine similarity in the embedding space.
- On the Scaling of PEFT: Towards Million Personal Models of Trillion Parameters
- FormulaOne: Measuring the Depth of Algorithmic Reasoning Beyond Competitive Programming
- Beyond Final Scores: A Systematic Evaluation of Agents for Long-Horizon AI Research and Development
- Value Leakage: An LLM's Answers Are Silently Shaped by Its Own Values
- Measuring Agents in Production
- LiveMCP-101: Stress Testing and Diagnosing MCP-enabled Agents on Challenging Queries
- AutoLab: Can Frontier Models Solve Long-Horizon Auto Research and Engineering Tasks?
- xDailyBench: Benchmarking LLMs on Professional Consultation for Real-Life Problems
Original note title
implicit requirement inference is a persistent bottleneck for everyday assistance — all 11 models trail by at least 9 points on implicit requirements