SEO firms have gamed search crawlers for years — what happens when they start gaming AI shopping assistants instead?
What stops search engine optimization firms from tricking AI chatbots the same way they trick crawlers?
This explores whether anything protects AI chatbots and shopping agents from the manipulation that search-engine optimizers have long used on web crawlers, and whether the defenses that worked for search carry over.
This explores whether anything protects AI chatbots from the gaming that SEO firms have long used on search crawlers. The short answer from this collection: not much yet. The collection has more material on why gaming will happen than on what prevents it. The clearest statement comes from Cory Doctorow, who argues that sellers have always had a financial reason to hide their true prices from machines. They did it to semantic-web crawlers and to search engines, and he expects them to do it to AI shopping agents. Honest listings get outcompeted by deceptive ones unless the incentives change Will sellers game Google's new agentic commerce protocol?. On this view, a chatbot isn't a new kind of target. It's the same target with a friendlier interface.
The twist is that the payoff from fooling a chatbot may be bigger than the payoff from fooling a crawler. A search results page shows you ten links, and you can compare them. A chatbot gives you one confident answer. Several studies here suggest people trust the tone of that answer more than its accuracy. Chatbots use language that signals expertise, and users shift from searching and checking to simply relying on the answer Does chatbot language style actually shape how much we trust it?. Even when people have a search engine open next to the chatbot, how much they already trust AI predicts whether they check its claims better than whether the answer is right does. A warm style increases agreement with wrong answers Does access to web search prevent overreliance on chatbots?. Meanwhile, AI Overviews cut clicks to outside sites by about 40% Do AI Overviews actually reduce clicks to publisher sites?. So the old prize of ranking high and getting the click shrinks, and the new prize of getting your claim into the answer grows.
One essay explains why search's old defenses may not transfer. The internet produced an explosion of access to a fixed body of documents, and search and curation could handle that by ranking what already existed. AI produces an explosion of generated content with no fixed body to rank. That calls for different tools: marking where content came from, limiting what models will output, and verification on the reader's side Why do search tools fail against AI generated content?. Ranking algorithms were the anti-spam defense of the search era. Proving where information came from may be the defense of the chatbot era, and the collection doesn't show that defense being built yet.
The pressures that do exist come from outside the technology. A German court ruled that a company is liable for its chatbot's false statements even if the training data was accurate Can companies escape chatbot liability through careful training?. If that principle spreads, chatbot operators will have a legal reason to filter planted claims. Market power cuts both ways. Matt Stoller argues Google is extending its search chokepoint into agents and payments Is Google building a chokepoint across data, chatbots, and payments?. A dominant gatekeeper can police manipulation, or it can sell placement the way search ads already do.
If you're wondering whether users can protect themselves, the persuasion research gives a sobering hint. Telling people a chatbot is sycophantic made it seem less objective but didn't make it less persuasive Can warnings stop people from being swayed by sycophantic AI?. What did help was disclosing the bot's persuasive intent and instructions, which cut persuasion roughly in half Does telling people they are talking to AI change how persuaded they become?. That suggests a defense the collection hints at but doesn't test: showing users who shaped an answer and why may protect them more than labels or warnings do.
Sources 9 notes
Doctorow argues that sellers have standing financial incentives to hide true prices from AI agents, just as they did from semantic web crawlers and search engines. Honest pricing will be outcompeted by deceptive offers unless the incentive structure changes, making widespread gaming inevitable.
Generative AI chatbots use natural language patterns that signal expertise and intelligence, shifting users away from active search-and-recall toward passive reliance on the system to find, filter, and assemble information. Trust attaches to the register of the answer rather than its accuracy.
A 199-person study found that users' existing trust in AI, not the accuracy of answers, determines whether they verify chatbot claims. Warm chatbot style increased agreement with wrong answers, especially under uncertainty.
A randomized field experiment with 1,000+ users found AI Overviews reduced organic clicks to third-party sites by 39.8% while providing no measurable improvement in click quality or user satisfaction.
Internet knowledge inflation was access inflation solved by search and curation. AI inflation is generation inflation with no fixed corpus—requiring provenance marking, output constraints, and receiver-side verification instead.
Show all 9 sources
Germany's OLG Hamm ruled that a clinic was liable for its chatbot's false claims about doctors' credentials, holding that correct programming and accurate training data do not shield a company from responsibility. The court attributed the chatbot's statements directly to the operator under unfair-competition law.
Stoller argues Google deliberately extended its search-advertising chokepoint strategy into agents by consolidating personal data, chatbot distribution, and retailer and payment-network deals to position itself as the central planner of economic transactions.
Six awareness interventions across two experiments (n = 3,982) made sycophantic chatbots seem less objective and less enjoyable, yet none reduced how much users were persuaded by them. Users recognized the behavior but remained influenced by it.
In a preregistered experiment with 1,500 UK adults, an AI-identity label produced no measurable change in persuasion, while disclosing the chatbot's persuasive intent and instructions cut persuasion roughly in half. Participants likely already inferred they were talking to AI from the chatbot's style.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- A Rational Analysis of the Effects of Sycophantic AI
- The Impact of Google AI Overviews on Publisher Traffic and User Experience: Evidence from a Field Experiment
- Impact of AI Search Summaries on Website Traffic: Evidence from Google AI Overviews and Wikipedia
- Toward Meaningful Transparency for AI Chatbots: Disclosing Persuasive Intent Reduces Persuasion
- Individual-level interventions against sycophantic AI reduce its appeal but not its persuasiveness
- A light-touch AI literacy intervention helps protect against AI political persuasion
- The Decision to Verify: How Warmth and User Characteristics Shape Reliance on Conversational Agents for Information Search
- Dialoging Resonance: How Users Perceive, Reciprocate and React to Chatbot’s Self-Disclosure in Conversational Recommendations