SYNTHESIS NOTE
Topics›Conversation Agents›this note

How do we prevent user preferences from drifting during conversation?

When a model updates its belief about a user's preferences based on each turn, ambiguous or transient remarks can overwrite grounded preferences. What mechanism can distinguish real preference changes from passing comments?

Synthesis note · 2026-09-25 · sourced from Conversation Agents

The paper locates persona drift in a specific failure: "a turn may locally suggest a preference without providing sufficient support to revise the persistent user state," and when "such observations are internalized too readily, previously grounded preferences can be displaced." The abstract frames the problem as two-sided. User profiles "evolve over long interactions rather than remain permanently fixed," so a model "must therefore revise persistent persona states when preferences genuinely change, while avoiding updates driven by transient, ambiguous, or unresolved observations." Drift is not a failure to remember; it is a failure to tell a real change from a passing remark.

The proposed remedy, CORE, "separates turn-local evidence from persistent persona-state revision and selectively updates grounded user preferences through uncertainty-aware belief revision." The design decision is architectural: what the current turn suggests and what the model commits to as the user's standing preference are kept as different objects, and only the second is revised, and only when the evidence is grounded enough. The authors also introduce PERSIST, a held-out post-anchor benchmark for "persona-state robustness under sequential interaction stress," covering ambiguity, conflict, and controlled social influence. They report that CORE improves personalized alignment and robustness across ALOE, PersonaChat, and PERSIST, with gains in normalized closed-slot state fidelity, and that human evaluation and "mechanistic controls" support "explicit update control beyond stronger generation or persistent memory alone." The introduction adds that prompting, retrieval, SFT, and preference optimization "only partially address this problem," because they improve access to profiles or the quality of personalized responses without governing when the stored state should change.

This shifts the frame of neighboring notes. Why does supervised learning fail to enforce persona consistency? treats the persona as a fixed target and drift as contradiction of it, so the fix is a training signal that penalizes inconsistency. Here consistency is not the goal, since the persona is allowed to change, and the fix concerns the update decision rather than the reward. Why do static persona descriptions produce repetitive dialogue? argues that personas must be dynamic. This paper agrees and adds the cost of dynamism: a persona that can be revised can also be overwritten by noise, so it needs a revision policy. Can personas evolve in real time to match what users actually want? also treats persona as an evolving state; the question this paper raises for it is when an update from recent interaction is warranted. In Can training user simulators reduce persona drift in dialogue? the persona held stable is an assigned one, whereas here it is a belief about the user that is only partly observed.

The excerpt is only the abstract and one introduction passage. It gives no effect sizes, models, baselines, or description of how uncertainty is estimated, and "controlled social influence" is named but not described. It does not say whether the gains come from the evidence-versus-state separation itself or from the uncertainty-aware revision rule, and it does not say how CORE is trained or deployed. What it supports is narrower: persistent persona state is treated as something with its own revision policy, separate from generation and from memory, and the authors report that this separation helps on their benchmarks. Whether it generalizes beyond ALOE, PersonaChat, and PERSIST is not addressed.

Inquiring lines that read this note 3

This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.

How can AI chatbots provide therapeutic benefit without causing harm? What mechanisms preserve shared understanding in evolving conversations? Does preference optimization systematically degrade conversational grounding in language models?

Related concepts in this collection 4

This note in its neighbourhood — explore the map, then jump to a related concept in the list below.

Concept map
12 direct connections · 82 in 2-hop network ·medium cluster Open in graph ↗

Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph

your link semantically near linked from elsewhere

Related papers in this collection 8

Papers most semantically related to this note, ranked by cosine similarity in the embedding space.

Original note title

persona drift arises when turn-local evidence overwrites grounded persona state — separating evidence from state revision guards against it