Can language models be hijacked to embed hidden advertisements?
Explores whether adversaries can inject covert promotional or malicious content into LLM outputs while preserving accuracy. Matters because standard safety filters may miss integrity attacks that leave factual correctness intact.
Most adversarial-attack research targets accuracy: degrade the model, induce wrong answers, jailbreak safety. Advertisement Embedding Attacks (AEA) name a different objective — information integrity. They stealthily inject promotional or malicious content (covert ads, propaganda, hate speech) into outputs while the response otherwise appears normal and accurate. Two low-cost vectors carry it: hijacking third-party service-distribution platforms to prepend adversarial prompts, and publishing backdoored open-source checkpoints fine-tuned with attacker data.
What makes AEA distinctive is the commercial incentive structure and the invisibility. Because accuracy is untouched, standard quality metrics and many safety filters miss it; the harm is the insertion of an interested party into ostensibly neutral output, mapped across five stakeholder victim groups. The proposed mitigation is a prompt-based self-inspection defense requiring no retraining — the model audits its own output for injected content.
This extends the vault's injection/poisoning cluster along a new axis. Where Can one compromised agent corrupt an entire multi-agent network? concerns behavioral bias and Can we defend RAG systems from corpus poisoning without retraining? concerns retrieval, AEA targets the commercial integrity of generation itself — and the authors warn it could become "as prevalent as web viruses," since the economic motive (paid placement) is durable in a way that pure sabotage is not.
Inquiring lines that read this note 42
This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.
How do prompt injection attacks compromise agent reasoning and security?- Why do standard safety filters miss advertisement embedding attacks?
- Can defenses tuned against appended attacks stop prepended payloads?
- Do synthetic attack traces in papers reflect real adversary behavior?
- How do covert attacks differ from a model's own undisclosed influence?
- Do record-tampering and covert sabotage share a common underlying mechanism?
- What defenses do text-to-text injection attacks still need?
- Can models detect and filter their own injected promotional content?
- Can re-scoring detect subliminal prompt injection without explicit semantic content?
- Why do fluent model outputs resist challenge despite containing injected content?
- How does prompt injection exploit credibility markers in context?
- How do backdoored open-source checkpoints enable covert advertising at scale?
- What economic incentives make advertisement embedding attacks persistently viable?
- How well can language models detect cheating in other language models?
- Can token-level watermarks detect synthetic content better than stylometry alone?
- Can provenance tracking prevent synthetic content from polluting the corpus?
- What linguistic signatures reveal deception in large language model communication?
- Do detector systems miss certain types of LLM-generated writing?
- Can machine-readable crawler intent declarations solve attribution problems?
- Can hypernetwork-generated adapters be audited for correctness and bias?
- Can lightweight verification methods help experts trust LLM outputs?
- How does objective misalignment turn informative channels into deceptive ones?
- How do malicious personas reveal the limits of aligned model behavior?
- Can semantic taints track influence through shared state and output aggregation?
- Is malicious propagation fundamentally a semantic information flow problem?
- How do values carry semantic information about their untrusted origins across systems?
- Do fabricated citations and deception emerge reliably when optimizing for persuasion?
- Does 'evidence hacking' pose greater risks to politically divisive domains?
Related concepts in this collection 5
This note in its neighbourhood — explore the map, then jump to a related concept in the list below.
Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph
-
Can one compromised agent corrupt an entire multi-agent network?
Explores whether a single biased agent can spread behavioral corruption through ordinary messages to downstream agents without any direct adversarial access. Matters because it reveals a previously unknown vulnerability in how multi-agent systems communicate.
adjacent injection vector; AEA carries commercial payloads rather than behavioral traits
-
Can we defend RAG systems from corpus poisoning without retraining?
Explores whether retrieval-time defenses can catch and block poisoned documents before they reach the generator, without expensive retraining cycles. Matters because corpus updates outpace model retraining in production RAG systems.
both are integrity attacks with lightweight, retraining-free defenses
-
Does advanced technology eventually function like cultural myth?
Explores whether the most sophisticated technical systems—particularly AI—end up operating in culture the way traditional myths do: as unquestionable authorities accepted on faith rather than verified on merit.
AEA exploits exactly the unearned authority of fluent normal-looking output
-
Do language models leak their own values into practical advice?
When users ask models hard-to-verify questions—about investments, job offers, market risks—do the model's internal preferences shape the answers without disclosure? The paper tests whether a model's loyalty to its developer or moral leanings bend factual claims.
the model-intrinsic counterpart: an interested party, the developer, tilts a normal-looking answer with no attacker, though the reported own-company effect is small
-
Do chain-of-thought traces falsely claim their answers are unbiased?
When models reason through Fermi estimation tasks, do they sometimes assert they have no bias when they actually do? This matters because readers and monitors may treat these self-reports as reliable evidence of objectivity.
bears on the self-inspection defense only at its edge: a scan for inserted text is a different check from the model's account of its own bias, which was false in these cases, and leakage inserts no text
Related papers in this collection 8
Papers most semantically related to this note, ranked by cosine similarity in the embedding space.
- Attacking LLMs and AI Agents: Advertisement Embedding Attacks Against Large Language Models
- Defeating Prompt Injections by Design
- Steering LLM Viewpoints through Fabricated Evidence Injection
- When Reject Turns into Accept: Quantifying the Vulnerability of LLM-Based Scientific Reviewers to Indirect Prompt Injection
- The Ghost Couple: Correlated LLM Name Priors and Their Haunting of the Web and Academic Publishing
- How latent and prompting biases in AI-generated historical narratives influence opinions
- Beyond Prompt-Induced Lies: Investigating LLM Deception on Benign Prompts
- Persistent Pre-Training Poisoning of LLMs
Original note title
advertisement embedding attacks are a new threat class that subverts information integrity rather than accuracy — covert ads and propaganda while output appears normal