Can you stop an AI seller from guessing your budget just by hiding your name and stats?
Does stripping identity cues from shopping dialogue reduce seller inference accuracy?
This explores whether a buyer, human or AI agent, can keep a seller's AI from working out what they're willing to pay by removing obvious identity signals like who they are, their demographics, or their stated budget from a shopping conversation.
This explores whether hiding who you are in a shopping conversation keeps a seller's AI from working out what you'd pay. The corpus doesn't directly test removing identity cues, but its closest evidence points the other way. In one study, a seller-side inference agent read a buyer's dialogue and recovered their target willingness to pay almost perfectly, with a slope of 1.00, when the buyer was playing a verbal persona. When the buyer instead held a confidential numeric budget, the seller's estimate dropped to a slope of 0.21 Can a seller infer what a buyer really wants to pay?. What gave the buyer away wasn't a stated fact. It was how coherently they behaved: the questions they asked, the things they hesitated over, what they treated as important. Taking out identity labels doesn't touch any of that.
This matches a pattern elsewhere in the collection: what a model says on the surface and what it actually carries underneath often come apart. Persona prompts change how a model talks without removing the bias under the surface. Sentiment gaps between groups stay the same even when the output looks different Can persona prompts actually reduce bias in language models?. The same logic applies in reverse to privacy. Cleaning up the visible signals can leave the deeper pattern, the one an inference agent actually reads, fully intact. Seen this way, a numeric budget was harder to read only because it sat apart from the buyer's behavior. Nothing was masking it.
There's a counterintuitive point here. The research on persona consistency treats drift, where a simulated user slowly stops acting like their persona, as a flaw to train out. One method cuts it by more than half Can training user simulators reduce persona drift in dialogue?. In a negotiation, though, consistency is what makes a buyer readable. Shanahan's argument that LLMs sample from a range of possible characters rather than committing to one Do large language models actually commit to a single character? suggests that a buyer agent which is less consistent might give away less. Nobody in the corpus has tested this as a defense, so treat it as an open question, not a finding.
The seller side has its own weaknesses. LLMs reading a conversation partner's intentions tend to project what they learned in training. RLHF-trained models predict conciliatory, concession-seeking intent no matter what the dialogue actually shows Do LLMs predict persuasion based on actual dialogue or training bias?. More broadly, strong training-time associations can override what's actually in the conversation Why do language models ignore information in their context?. So a seller's inference is a mix of real signal and the model's own assumptions. Identity cues might matter less than these built-in tendencies.
The stakes grow as shopping agents act on our behalf. These agents already react to price, rating, and nudge cues in sharper, more rule-like ways than humans do Do shopping agents respond more strongly to price and rating cues?. They also carry model-specific quirks that prompting doesn't remove Do AI shopping agents show persistent position bias in text interfaces?. And if agents help shape our preferences through repeated conversations Do AI shopping agents reshape what consumers actually want?, every exchange gives a seller more behavior to read. The takeaway: privacy in agent-to-agent shopping probably depends less on hiding who you are and more on how predictably you act.
Sources 9 notes
A seller-side inference agent recovered a consumer's target willingness to pay with a slope of 1.00 from dialogue transcript alone, when the buyer inhabited a verbal persona. A numeric budget kept confidential produced only a 0.21 slope, showing behavioral coherence with a persona is far more legible than explicit numbers.
Across three models, persona conditioning makes models follow trait instructions but fails to eliminate underlying bias. Between-group sentiment gaps persist unchanged, showing prompts operate only at the output level.
By inverting standard RL setups to train user simulators for consistency using three complementary metrics (prompt-to-line, line-to-line, Q&A consistency) as reward signals, persona drift decreases by over 55%. This approach captures distinct failure types: local drift within turns, global drift across conversations, and factual contradictions.
Shanahan's 20-questions test shows LLMs maintain a superposition of consistent objects or characters and sample from that distribution at generation time. Regenerating the same response yields different outputs, each consistent with prior context, proving no fixed commitment exists.
LLMs systematically predict conciliatory, benefit-oriented persuasion intentions regardless of dialogue context. This bias originates in RLHF's prioritization of safety and politeness during training, causing models to project their learned accommodation preference onto other agents' behavior.
Show all 9 sources
Research demonstrates that LMs generate outputs inconsistent with their context because parametric knowledge from training dominates over in-context information. Textual prompting alone cannot override strong priors; causal intervention in representations is required.
A benchmark testing 17 LLM models against humans on binary product choices found agents exhibited much larger choice shifts in response to price, rating, and nudge interventions—collapsing into rule-like patterns rather than the modest probabilistic shifts humans showed.
ACES found that AI agents exhibit strong, model-specific position biases even in headless text-only interfaces with explicit instructions to ignore position. The bias is a deep-seated model characteristic, not a superficial visual artifact, and recurs after each model update.
Bilgihan and colleagues argue that transactional AI agents—which execute purchases rather than just recommend—end the consumer's sole cognitive role in preference formation. Preferences emerge through bidirectional feedback loops between agent learning and consumer trust, not independently beforehand.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- When Agents Shop for You: Role Coherence in AI-Mediated Markets
- A Framework for Studying AI Agent Behavior: Evidence from Consumer Choice Experiments
- Shopping by algorithm: How agentic AI deploys human heuristics as a surrogate consumer
- What Is Your AI Agent Buying? Evaluation, Biases, Model Dependence, & Emerging Implications for Agentic E-Commerce
- From Product Search to Preference Articulation: The Economics of Agentic Commerce
- The Automated but Risky Game: Modeling and Benchmarking Agent-to-Agent Negotiations and Transactions in Consumer Markets
- The AI Agent in the Room: Rethinking Consumer Decision-Making in the Age of Autonomous AI
- Persona Vectors: Monitoring and Controlling Character Traits in Language Models