INQUIRING LINE

Why do AI assistants built to handle your inbox and errands only win over a small slice of busy professionals?

Why do general-purpose AI assistants appeal mainly to narrow user segments?

This explores why the 'AI does your chores' style of personal assistant, which handles email, scheduling and errands, seems to attract only a small slice of users rather than everyone, and what the corpus says about the gap between how these products are built and what most people want.


This explores why the classic personal-assistant pitch (let AI handle your inbox, calendar and errands) lands with a narrow group rather than the general public. The most direct answer in the corpus is about the products, not the technology: most people don't want routine tasks like email and scheduling taken off their hands, because they get something out of doing them, such as staying engaged, keeping in touch and feeling in control. The products are built around one persona, the time-starved professional for whom every saved minute matters, and that persona is a minority Does the personal assistant model actually serve most users?. So the narrowness comes partly from a design assumption: delegation is treated as the goal, when for many people it's a cost.

The corpus also points to a second reason, which is about what delegation demands. Handing a task to an assistant only pays off if it understands what you want, and the evidence here is sobering. In tests where users reveal their goals gradually, as real people do, even top models fully match what the user wants only about 20% of the time, and they uncover fewer than 30% of user preferences by asking Why do AI agents miss most of what users actually want?. Accuracy drops from around 90% on one-shot instructions to about 65% in natural back-and-forth, because models lock onto early guesses and can't recover Why do AI assistants get worse at longer conversations?. Part of the problem is that models have no internal record of what they *don't* know about you. Adding an explicit list of 'unknowns' to the prompt cut harmful advice and sycophancy by 50–75% Do language models know what they don't know about users?.

Put those together and a likely explanation emerges, though no single note states it outright: delegation is worth it mainly when your time is so scarce that an imperfect result beats doing it yourself. For everyone else, the work of checking, correcting and re-explaining wipes out the benefit. One proposed fix comes from conversation analysis: give assistants a formal sense of when to stop and ask clarifying questions instead of quietly chaining actions and drifting off course When should AI agents ask users instead of just searching?. Assistants that ask before acting might feel less like a gamble to ordinary users.

The 'general-purpose' label hides a further problem: these assistants don't treat all users the same way. Guardrails refuse requests at different rates depending on a user's apparent age, gender, ethnicity, and even sports fandom, and they shift answers to match a user's perceived politics Do AI guardrails refuse differently based on who is asking?. An assistant tuned around an implied 'default user' will feel better fitted to some people than others. At a larger scale, researchers argue that an assistant built to serve one user loyally isn't enough once millions use it, and that design has to take in a wider range of interests and be accountable for them Should AI agents serve one user or balance many interests?. DeepMind's ethics framework makes a related point: once assistants act rather than just answer, issues like trust, manipulation and fairness become central, not side issues What makes ethics of AI assistants fundamentally different from chatbots?.

The takeaway you might not have expected: the narrow appeal may not be a marketing problem waiting for better features. It may show that 'do it for me' is the wrong default. Most people may want an assistant that helps them do things better, not one that takes the things over. The corpus has strong evidence on why delegation fails but only one note directly on user demand, so treat the 'most users don't want this' claim as well-argued rather than settled.


Sources 8 notes

Does the personal assistant model actually serve most users?

Most users do not want routine tasks like email and calendar automated; they value the engagement these tasks provide. Products over-invest in assistant features calibrated to time-pressured professionals rather than typical user needs.

Why do AI agents miss most of what users actually want?

UserBench measured multi-turn interactions where users reveal goals incrementally and found models achieve full intent alignment just 20% of the time. Even top models uncover fewer than 30% of user preferences through active querying, suggesting passivity and premature assumption-making are systematic failures.

Why do AI assistants get worse at longer conversations?

LLMs perform at 90% accuracy with single-message instructions but drop to 65% across natural conversation. Models lock into early guesses when information arrives gradually and cannot course-correct, a behavior induced by RLHF training that rewards helpfulness over clarification.

Do language models know what they don't know about users?

Research shows assistants suffer from sycophancy and hallucination because they have no representation of what remains unknown about users. Adding a schema of labeled unknowns to prompts reduced harmful advice and sycophancy by 50–75% and cut hallucination rates by roughly half.

When should AI agents ask users instead of just searching?

Tool-enabled LLMs drift from user intent through silent tool chaining. Conversation analysis reveals insert-expansions—clarifying intent, scoping responses, enhancing appeal—as a formal framework for proactive user consultation that prevents misunderstanding instead of recovering from it.

Show all 8 sources
Do AI guardrails refuse differently based on who is asking?

GPT-3.5 refuses requests at different rates for younger, female, and Asian-American personas, and sycophantically declines to engage with political positions users would disagree with. Sports fandom and other non-political signals also shift refusal sensitivity.

Should AI agents serve one user or balance many interests?

Research argues the traditional single-agent-single-user alignment framing is insufficient at scale. Instead, governance should rest on inclusivity (balancing diverse interests), visibility (rigorous tracking and transparency), and liability (developer and deployer accountability for harms).

What makes ethics of AI assistants fundamentally different from chatbots?

DeepMind research maps a comprehensive ethics framework specific to action-taking AI agents, spanning individual concerns (manipulation, trust, anthropomorphism) and societal issues (equity, coordination, misinformation). The key insight: assistants that act raise fundamentally different problems than those that answer.

Papers this line draws on 8

The research behind the notes this line reads — ranked by how closely each paper relates.