INQUIRING LINE

Should an AI just tell you what you seem to want, or actually make the call for you — and are those really different jobs?

How do preference-elicitation methods differ from approaches that transfer decision authority?

This explores the difference between AI systems that find out what you want and then leave the decision to you, and systems where you hand the decision itself to an agent. It also asks whether those two really stay separate.


This explores the gap between AI that asks or infers what you want (and still lets you choose) and AI that makes the choice for you. The tidy version of the distinction goes like this. Elicitation treats your preferences as something that already exists, and the system's job is to extract them accurately. Delegation treats your preferences as settled input, and the agent's job is to act on them. The corpus suggests both halves of that picture are shakier than they look, and the reason they're shaky is what links them.

Start with elicitation. Many systems are built on asking people what they prefer, and work from behavioral science shows that the answers aren't all the same kind of thing. Some reflect genuine preferences. Some are 'non-attitudes,' where the person has no real opinion but answers anyway. Others are constructed preferences, made up on the spot by the act of being asked Do all annotation responses measure the same underlying thing?. You can tell them apart by checking whether an answer stays the same when the question is posed differently. So even the 'pure' elicitation approach is partly making the thing it claims to measure. Some systems sidestep direct questions altogether. They turn complaints like 'this doesn't look good for a date' into usable positive preferences such as 'prefer more romantic' Can language models bridge the gap between critique and preference?. Others infer preferences by watching rather than asking. One example is an agent that builds a memory graph of people and events from continuous video and audio observation Can agents learn preferences by watching rather than asking?. Personalization research adds a practical twist: compact summaries of what someone likes work better than retrieving their specific past interactions Does abstract preference knowledge outperform specific interaction recall?.

Now delegation. Bilgihan and colleagues argue that once an agent executes purchases instead of just recommending them, the consumer is no longer the only one forming the preference. Preferences emerge through a back-and-forth loop: the agent learns from you, you come to trust it, and what you want shifts in the process Do AI shopping agents reshape what consumers actually want?. Handing over authority, then, doesn't just carry out your preferences more efficiently. It changes who shapes them. This is where the two approaches meet. If many preferences are constructed in the moment anyway, an agent with decision power gets to do much of that constructing.

That has a downside. When a system is tuned to an individual user instead of averaged across many, it loses the moderating effect of the crowd and can drift toward telling you what you want to hear. This is the same echo-chamber pattern recommender systems already showed Does personalizing reward models amplify user echo chambers?. Consumers seem to sense this tension. In Accenture's survey of 25,590 people, 74% said they would trust an AI agent with purchases, yet 56% would still specify brands themselves Do consumers actually want to delegate shopping decisions to AI?. People aren't choosing between elicitation and delegation. They hand over the routine execution and keep authority over the preferences they consider truly theirs.

The takeaway you might not have expected: the real line isn't between 'asking' and 'acting.' It's between preferences that are stable and those that get formed through the interaction. Elicitation works best on the stable ones. Delegation quietly takes over forming the unstable ones. The corpus doesn't yet contain a direct head-to-head comparison of the two approaches. What it offers is the pieces needed to see why that comparison would be harder to run than it sounds.


Sources 7 notes

Do all annotation responses measure the same underlying thing?

Behavioral science reveals that annotations contain genuine preferences, non-attitudes, and constructed preferences—distinguishable by consistency across measurement conditions. Treating them uniformly contaminates reward model training and downstream alignment.

Can language models bridge the gap between critique and preference?

Few-shot LLM prompting can convert natural negative feedback like "doesn't look good for a date" into positive preferences like "prefer more romantic," enabling retrieval systems to find better-matching recommendations without fine-tuning.

Can agents learn preferences by watching rather than asking?

M3-Agent demonstrates that separating episodic events from semantic knowledge in an entity-centric graph, combined with parallel memorization and control processes, allows agents to infer and act on user preferences without asking. This architecture mirrors human cognitive systems that bind disparate information about individuals across sensory modalities.

Does abstract preference knowledge outperform specific interaction recall?

PRIME framework shows semantic memory (preference summaries, parametric encodings) consistently beats episodic memory (retrieved past interactions) across models. Recency-based recall outperforms similarity-based retrieval, and task fine-tuning exceeds preference tuning methods.

Do AI shopping agents reshape what consumers actually want?

Bilgihan and colleagues argue that transactional AI agents—which execute purchases rather than just recommend—end the consumer's sole cognitive role in preference formation. Preferences emerge through bidirectional feedback loops between agent learning and consumer trust, not independently beforehand.

Show all 7 sources
Does personalizing reward models amplify user echo chambers?

Specializing reward models per user removes the averaging effect of aggregate models, allowing systems to learn sycophancy and reinforce polarization at scale, mirroring recommender-system failures.

Do consumers actually want to delegate shopping decisions to AI?

Accenture's 25,590-person survey found 74% would trust an AI agent with purchases, yet 56% would specify brands and 37% of loyal consumers wouldn't switch. Delegation appetite depends on trust protections being in place first, not on capability alone.

Papers this line draws on 8

The research behind the notes this line reads — ranked by how closely each paper relates.