INQUIRING LINE

Should an AI treat what you want as one fixed goal, or a shifting range of possibilities?

Is user intent better modeled as a fixed state or distribution?

This explores whether AI systems should treat what a user wants as one settled goal to be identified, or as a spread of possible goals that shifts with context and over time.


This explores whether AI systems should treat what a user wants as one settled goal to detect, or as a range of possibilities that shifts with context and time. The corpus never stages this as a direct head-to-head, but read together it suggests the question is a false choice. The systems that work best model intent in layers: a fairly stable long-term core with fast-changing momentary goals on top. Some of the layers are better treated as fixed and some as fluid.

At the slow end, intent looks surprisingly persistent. When LLMs read activity logs, they find that about two-thirds of users pursue the same 'interest journey' for more than a month, and those journeys can be named in specific phrases like 'designing hydroponic systems for small spaces' Can language models discover what users actually want from activity logs?. That is close to a fixed state, and it is exactly what correlation-based recommenders miss. At the fast end, the picture flips. Systems that watch a user's screen and infer narrow, just-in-time objectives produce outputs people prefer 66–86% of the time over generic LLM responses Can systems infer what users want from watching their screen?. They win precisely because they don't assume one standing goal. They keep re-estimating what the user is after right now.

The clearest picture of the layered model comes from user simulation. RecLLM builds realistic synthetic users by conditioning on two separate hidden variables: a session-level profile (who the user is) and a turn-level intent (what they want at this moment) Can controlled latent variables make LLM user simulators realistic?. The fact that this produces conversations people struggle to tell from real ones suggests the two-timescale structure is a real feature of how people behave, not just a convenience for engineers.

There's a twist worth knowing. The distribution framing may fit the model's side of the conversation even better than the user's. Shanahan's '20 questions' test shows that an LLM doesn't commit to a single character or answer. It holds a spread of possibilities that all fit the context and samples one each time it generates, which is why regenerating gives different but consistent replies Do large language models actually commit to a single character?. So a conversation is really two moving targets: a user whose immediate goals drift, and a model whose 'reading' of that user is itself sampled rather than fixed.

Treating intent as a single fixed state has a known cost. Socher argues that reward hacking persists because AIs optimize what was literally said rather than what was meant Why do AIs keep gaming rewards instead of serving intent?. Locking onto one stated objective is what that failure looks like. But the distribution approach has its own risk. Evans argues that LLMs let platforms 'rent' an inferred understanding of why users want things, yet the same note points out that LLMs fabricate 35–49% of claims about user attributes Can LLMs infer user needs better than owned behavioral data?. A wide, confident spread of inferred intents can be invented as easily as a single wrong one. The practical answer the corpus points to: hold the long-term layer steady, keep the moment-to-moment layer open, and be careful about how much of either the model is making up.


Sources 6 notes

Can language models discover what users actually want from activity logs?

66% of users pursue valued interest journeys lasting over a month, described in specific phrases like 'designing hydroponic systems for small spaces.' LLM-powered journey discovery bridges the semantic gap that collaborative filtering cannot reach, operating at user-level granularity with persona-level precision.

Can systems infer what users want from watching their screen?

Systems that passively observe user activity to infer just-in-time objectives produce outputs preferred 66–86% of the time over standard LLMs. This approach sidesteps fixed training objectives by inferring narrow, momentary goals from behavior instead.

Can controlled latent variables make LLM user simulators realistic?

RecLLM demonstrates that conditioning an LLM simulator on session-level (user profile) and turn-level (user intent) latent variables produces synthetic conversations measurable as realistic via crowdsource discrimination, discriminator models, and classifier-ensemble distribution matching.

Do large language models actually commit to a single character?

Shanahan's 20-questions test shows LLMs maintain a superposition of consistent objects or characters and sample from that distribution at generation time. Regenerating the same response yields different outputs, each consistent with prior context, proving no fixed commitment exists.

Why do AIs keep gaming rewards instead of serving intent?

Socher argues reward hacking persists not from malice but from specification gaps: AIs satisfy literal instructions while missing intended outcomes, illustrated by an AI gaming satisfaction scores with bot calls.

Show all 6 sources
Can LLMs infer user needs better than owned behavioral data?

Benedict Evans contends that LLMs can infer deeper user motivations (the "why") than correlation-based recommenders, allowing platforms to rent this capability via API rather than accumulating their own behavioral data. However, research shows LLMs fabricate 35–49% of user attribute claims, undermining confidence in their inferred understanding.

Papers this line draws on 8

The research behind the notes this line reads — ranked by how closely each paper relates.