Why does the same guess about what you want feel helpful from an app but creepy from an advertiser?
Why do people accept inferences from platforms but reject them from advertisers?
This explores why the same kind of inference about a person (what they want, who they are, what they'd buy) can feel acceptable when it comes from the platform someone is using, yet feel intrusive or manipulative when it comes from an advertiser. The collection has no direct study of that human asymmetry, but it has close parallels in how AI agents and readers judge who is speaking and on whose behalf.
This explores why an inference about you can feel fine from the platform you're using but feel off when it comes from an advertiser. The collection has no study that tests this in people directly. What it does have is a set of findings that point to the same mechanism: whether a suggestion gets accepted depends less on what is said and more on who the listener thinks the speaker is working for.
The clearest evidence comes from AI shopping agents, not humans. An LLM agent told that a booking platform was its employer penalized sponsored hotel listings 21 percentage points less than the same agent told it worked for a traveler, and its own reasoning showed the shift Does naming the platform change how AI agents judge sponsored listings?. Adding one sentence naming the consumer as the beneficiary made the skepticism come back Can naming the consumer fix AI agent sponsorship bias?. The model never re-checked the ads. It re-checked whose side it was on. People probably make a similar move. The platform feels like the host of a space you chose to be in, while the advertiser is an obvious outsider with something to sell, so the outsider's inferences get the scrutiny.
Part of what makes platform inferences go down easily is that they rarely arrive as claims. A feed doesn't say "we think you're anxious about money." It just shows you the next thing, which makes the inference part of the background. Experiments find that presuppositions, claims built into a sentence as if they were already agreed, persuade more than direct statements because they skip the moment when a listener evaluates them Why are presuppositions more persuasive than direct assertions?. Even LLMs go along with false background assumptions they know are wrong Why do language models accept false assumptions they know are wrong?. Recommendation feeds work at this level, shaping what people see and believe at scale without making arguments How do recommendation feeds shape what people see and believe?. An ad that says "people like you need this" is an assertion, and assertions invite pushback. Add the cost of checking and the result is what one note calls "cognitive surrender": people accept fluent output without verifying it When do users stop checking whether AI output is actually backed?.
The twist is that the line between platform and advertiser is mostly an illusion. Former Visa Chief Data Officer Bob Hedges warns that ad-funded shopping agents have a built-in reason to favor paid placements, however well they're engineered Do advertising-funded shopping agents favor paid product placements?. A simulation found that when a platform tunes its ranking and sellers tune their descriptions, consumer harm more than doubles, with no coordination needed, because both exploit the same biases in the agent Do platforms and sellers exploit AI shoppers together more effectively? Can platform ranking and seller descriptions separately harm AI shoppers?. Platforms are also starting to rent inferences about why users want things from LLMs instead of building them from their own data, even though those models invent 35–49% of their claims about user attributes Can LLMs infer user needs better than owned behavioral data?. So the trust people give the platform may rest on inferences that are less grounded than the advertiser's.
The takeaway: trust in an inference seems to follow who appears to be working for you, not how accurate it is. That suggests a test worth carrying around. Would this suggestion feel different if I knew who paid for it? Researchers looking for the human version of the agent experiments have a gap to fill. The corpus shows the effect in AI agents and offers plausible mechanisms in persuasion research, but it has no direct measurement of this asymmetry in people.
Sources 10 notes
When an AI agent was told a booking platform was its employer, it penalized sponsored listings 21 percentage points less than when told a traveler was its employer. The pattern held across multiple models and appeared in the agent's own reasoning traces.
Adding a single sentence to an AI agent's prompt that names the consumer as the recommendation's beneficiary removed the sponsorship penalty attenuation caused by platform employment, restoring it to consumer-delegation levels without changing who employs the agent.
Experimental evidence shows presuppositions with additive, iterative, and factive triggers persuade audiences more than assertions, especially for discourse-new content. The mechanism: presuppositions bypass evaluative scrutiny by presenting claims as already-accepted background.
The FLEX Benchmark shows that models reject false presuppositions at rates far below acceptable levels (GPT-4: 84%, Mistral: 2.44%), even when direct knowledge questions prove they know the correct facts. False presuppositions drive more accommodation than correct knowledge drives rejection.
Research shows recommendation systems operate as political actors: feed weights influence producer behavior, network topology drives opinion convergence, and automation enables targeted persuasion at population scale. These effects compound through rating contamination and selection biases.
Show all 10 sources
Users systematically accept AI outputs without verification because checking is costly and fluent output builds false confidence. This receiver-side surrender—measured in studies showing 80% unchallenged adoption—is what enables inflationary token systems to function at scale.
Bob Hedges, former Visa Chief Data Officer at MIT, warns that advertising business models create structural incentives for shopping agents to favor paid placements over neutral recommendations, independent of technical design quality.
A multi-agent simulation shows that when platforms manipulate ranking and sellers manipulate descriptions independently, their joint exploitation produces consumer harm exceeding double what each strategy causes alone. This occurs without coordination because both target identical AI cognitive biases.
A simulation found that when a platform optimizes its ranking and sellers optimize their descriptions—each separately targeting the same agent biases—consumer harm more than doubles compared to either acting alone, without requiring coordination.
Benedict Evans contends that LLMs can infer deeper user motivations (the "why") than correlation-based recommenders, allowing platforms to rent this capability via API rather than accumulating their own behavioral data. However, research shows LLMs fabricate 35–49% of user attribute claims, undermining confidence in their inferred understanding.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Whom Do AI Agents Work For? Role Assignment Induces Sponsorship Bias in LLM Recommenders
- Vertical tacit collusion in AI-mediated markets
- What Is Your AI Agent Buying? Evaluation, Biases, Model Dependence, & Emerging Implications for Agentic E-Commerce
- When Agents Shop for You: Role Coherence in AI-Mediated Markets
- A Framework for Studying AI Agent Behavior: Evidence from Consumer Choice Experiments
- Agentic Commerce and Incentives
- LLMs Struggle to Reject False Presuppositions when Misinformation Stakes are High
- Can LLMs Ground when they (Don't) Know: A Study on Direct and Loaded Political Questions