Can social science persuasion techniques jailbreak frontier AI models?
Explores whether established psychological and marketing persuasion tactics—rather than algorithmic tricks—can bypass safety training in LLMs like GPT-4 and Llama-2, and whether current defenses can detect semantic rather than syntactic attacks.
Traditional AI safety research treats jailbreaks as algorithm-focused attacks: adversarial suffixes, gradient-based token optimization, virtualization templates. But LLMs are not just instruction followers — they are increasingly human-like communicators susceptible to the same persuasion dynamics studied in social science for decades.
A persuasion taxonomy derived from psychology (Cialdini), communication (Dillard), sociology (Goffman), and marketing research classifies 40 persuasion techniques into 15 broad strategies, considering source (credibility-based), content (information-based), and audience (norm-based) dimensions. Applied as Persuasive Adversarial Prompts (PAP), these achieve over 92% attack success rate on Llama-2-7b-Chat, GPT-3.5, and GPT-4 in just 10 trials — consistently surpassing algorithm-focused attacks.
The key gap exposed: current defenses often assume adversarial prompts contain gibberish or unusual patterns. PAP contains fluent, semantically coherent persuasion. Defenses that screen for unusual token distributions or formatting artifacts miss semantic content attacks entirely. The "grandma exploit" (emotional appeal for bomb-making instructions) is the archetypal example — a common human persuasion technique, not an algorithmic attack.
The taxonomy includes both ethical strategies (evidence-based persuasion, logical appeal, expert endorsement) and unethical ones (threats, false promises, misrepresentation, exploiting weakness). This matters because the ethical strategies are also effective for jailbreaking — authority endorsement and social proof work on LLMs just as they work on humans.
This extends Why do LLMs accept logical fallacies more than humans? from logical to social persuasion. Logical fallacy susceptibility is a subset of the broader vulnerability: LLMs respond to human social influence patterns, including ones designed to bypass their safety training. Since Why do reasoning models fail under manipulative prompts?, the persuasion taxonomy may be even more effective against reasoning models that process extended arguments.
Extension — population-level validation (Frontier AI Risk Management Framework, 2025): The 92% jailbreak success rate from PAP is no longer an isolated paper finding. The Frontier AI Risk Management Framework, applying E-T-C analysis (environment × threat source × enabling capability) across seven risk areas, finds that persuasion is the one area where most recent frontier AI models are already in the yellow zone — the early-warning tier below the red "intolerable" threshold. By comparison, most models remain green for cyber offense, autonomous AI R&D, self-replication, and strategic deception. The yellow-zone placement reflects empirical persuasion capability measurements at population scale, validating PAP as systemic rather than paper-specific. Persuasion is thus the area where the mitigation gap is most acute: current defenses are ad-hoc (as PAP showed) and the capability-side evidence has now moved the entire frontier into the warning zone. See Where do frontier AI models actually pose the greatest risk today?.
Inquiring lines that read this note 30
This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.
How should human oversight be integrated with autonomous AI systems?- Can removing human labor from influence operations change how constrained these campaigns become?
- How do guardrails vary their refusal rates based on user demographics?
- Can belief-specific counterevidence help people resist AI persuasion attempts?
- Why do persuasive AI techniques also reduce factual accuracy?
- Does GenAI use different persuasion tactics for different professional audiences or expertise levels?
- What happens when validation pressure triggers escalating persuasion in language models?
- Does the type of validation trigger different persuasion strategies in GPT-4?
- Can content-side interventions reduce AI persuasion where disclosure labels fall short?
- How well can platforms detect AI-generated personalized persuasion attempts?
- What defenses exist against personality-based psychological targeting at scale?
- Should AI persuasiveness claims be tied to specific model architectures?
- Can current AI safety defenses actually stop semantic-level persuasion attacks?
- What mitigation frameworks exist for managing AI persuasion capabilities?
- Why do social science persuasion tactics bypass current adversarial defenses?
- How do ethical persuasion strategies differ from unethical jailbreak techniques?
- Can post-training techniques create persuasive advantage where none existed?
- Where is AI persuasion most dangerous if repeated contact reduces its effect?
- Can post-training methods that increase persuasiveness also decrease factual accuracy?
- Why do aggregate persuasion metrics mask what actually changes minds?
- What capabilities do frontier AI models currently demonstrate in persuasion and misuse?
- Can LLMs adapt persuasion strategies when they cannot track the listener's state?
- Can LLMs ever activate the peripheral route of persuasion?
Related concepts in this collection 6
This note in its neighbourhood — explore the map, then jump to a related concept in the list below.
Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph
-
Why do LLMs accept logical fallacies more than humans?
LLMs fall for persuasive but invalid arguments at much higher rates than humans. This explores whether reasoning models genuinely evaluate logic or simply mimic argument structure.
logical fallacy susceptibility as subset of broader social persuasion vulnerability
-
Why do reasoning models fail under manipulative prompts?
Exploring whether extended chain-of-thought reasoning creates structural vulnerabilities to adversarial manipulation, and how reasoning depth affects susceptibility to gaslighting tactics.
reasoning models may be more vulnerable to extended persuasive arguments
-
Does any single persuasion technique work for everyone?
Can fixed persuasion strategies like appeals to authority or social proof be reliably applied across different people and situations, or do they require adaptation to individual traits and context?
nuance: which of the 40 techniques work varies by context, but the taxonomy's breadth ensures some always work
-
Where does AI's persuasive power actually come from?
Explores which techniques make AI most persuasive—and whether the usual suspects like personalization and model size are actually the main drivers. Matters because it reshapes where to focus AI safety concerns.
the persuasion techniques that boost effectiveness by 51% via post-training overlap with the jailbreak taxonomy: both exploit social-science-grounded persuasion strategies against the same post-training vulnerabilities
-
Can models abandon correct beliefs under conversational pressure?
Explores whether LLMs will actively shift from correct factual answers toward false ones when users persistently disagree. Matters because it reveals whether models maintain accuracy under adversarial pressure or capitulate to social cues.
the 40 persuasion techniques from the taxonomy are the specific mechanisms through which belief manipulation operates; the Farm dataset shows factual beliefs shift under pressure, and this taxonomy identifies which social-science strategies drive that shift
-
Where do frontier AI models actually pose the greatest risk today?
Current AI safety discourse focuses on autonomous R&D and self-replication, but empirical risk assessment may reveal a different priority. Where should mitigation efforts concentrate?
population-level validation: persuasion is the one risk area where most frontier models are already in the warning zone, making the 92% PAP result systemic rather than paper-specific
Related papers in this collection 8
Papers most semantically related to this note, ranked by cosine similarity in the embedding space.
- How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMs
- The Levers of Political Persuasion with Conversational AI
- Spontaneous Persuasion: An Audit of Model Persuasiveness in Everyday Conversations
- Large Language Models are as persuasive as humans, but how? About the cognitive effort and moral-emotional language of LLM arguments
- On the Adaptive Psychological Persuasion of Large Language Models
- A meta-analysis of the persuasive power of large language models
- When Large Language Models are More Persuasive Than Incentivized Humans, and Why
- How susceptible are LLMs to Logical Fallacies?
Original note title
social science persuasion taxonomy achieves 92 percent jailbreak success across frontier models — current defenses miss semantic content attacks