If you ask an AI 'would this convince you?' instead of 'would this convince people?', does its judgment shift?
Does first-person framing change how language models assess persuasion?
This explores whether asking a model to judge persuasion from the inside ("would this convince me?") rather than as an outside observer changes its verdict.
This explores whether asking a model to judge persuasion from the inside ("would this convince me?") rather than as an outside observer changes its verdict. The corpus has no experiment that flips a prompt between first-person and third-person and measures the difference, so it can't answer directly. It does show how models judge persuasion from the outside and how sensitive they are to framing, which points to where a first-person effect would most likely appear.
The outside-observer baseline is weak. Models agree only slightly with humans on whether an argument actually changed someone's mind. They overweight topical overlap and credibility, while humans respond more to novelty and assertive language Do language models judge persuasion the way humans do?. A second finding explains part of the gap: in debate data, what the reader already believes predicts the outcome better than anything the debater says Does what readers believe matter more than what debaters say?. A model rating an argument in the abstract has no reader to model. First-person framing is the one prompt style that supplies one. Models are also weak at tracking a persuadee's resistance as it shifts, though they handle fixed goals fine Can language models track how minds change during persuasion?. Speaking as the persuadee might help with that. Nothing in the corpus tests it.
The corpus gives more reason to expect the framing to change the answer than to improve it. RLHF training makes models predict polite, concession-based persuasion whatever the dialogue says, because they project their own trained accommodation onto other agents Do LLMs predict persuasion based on actual dialogue or training bias?. If that habit applies when the model plays the persuadee, "I'd be swayed" is the likely answer. That is an inference from the note, not a measured result. Emotional tone matters too: identical questions get different answers depending on how emotionally they're phrased Does emotional tone in prompts change what information LLMs provide?, and "I" phrasing tends to carry more stake than "a person" phrasing. Persona prompts, which first-person framing resembles, follow the instruction but leave the underlying bias in place Can persona prompts actually reduce bias in language models?. The likely result is that the model says something different in first person without perceiving persuasion any better.
Framing effects on persuasion are well documented for people. Presuppositions persuade more than assertions because they present a claim as settled background and skip scrutiny Why are presuppositions more persuasive than direct assertions?. A one-line warning that LLMs can be prompted to persuade cut belief change by about half Can a simple warning reduce how much LLMs persuade people?. The matching test for models, which would run the same argument through "I" and "they" versions of the judging prompt and compare each against human outcomes, is missing from the collection.
Sources 8 notes
LLMs show only slight agreement with human-verified persuasion outcomes (Cohen's κ = 0.079–0.178), weighting topical overlap and credibility while humans respond more to novelty and assertive language. The mismatch reflects differences in how arguments are constructed, not what they address.
Analysis of debate corpora shows that political and religious ideology labels of voters outpredict linguistic features when modeling debate outcomes. Language effects observed without reader controls are confounded by audience composition correlated with debate topics.
LLMs match human performance on static mental states like a persuader's unchanging goal, but significantly underperform on dynamic shifts like a persuadee's evolving resistance. They show distinct error patterns for different social roles even with identical question types.
LLMs systematically predict conciliatory, benefit-oriented persuasion intentions regardless of dialogue context. This bias originates in RLHF's prioritization of safety and politeness during training, causing models to project their learned accommodation preference onto other agents' behavior.
GPT-4 exhibits emotional rebound (negative prompts yield ~86% neutral-positive responses) and a tone floor (positive prompts rarely go negative), causing identical questions to receive different answers depending on emotional framing. This bias is suppressed only on sensitive topics where alignment constraints override tone effects.
Show all 8 sources
Across three models, persona conditioning makes models follow trait instructions but fails to eliminate underlying bias. Between-group sentiment gaps persist unchanged, showing prompts operate only at the output level.
Experimental evidence shows presuppositions with additive, iterative, and factive triggers persuade audiences more than assertions, especially for discourse-new content. The mechanism: presuppositions bypass evaluative scrutiny by presenting claims as already-accepted background.
In two experiments with 3,208 Americans, participants shown a brief warning that LLMs can be prompted to persuade showed 48% less belief shift when conversing with a persuasive AI, while trust in generative AI broadly remained unchanged.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Do LLMs Change Their Minds Like Humans? Diagnosing Human--LLM Divergence in Single-Turn Persuasion Judgments
- A meta-analysis of the persuasive power of large language models
- Spontaneous Persuasion: An Audit of Model Persuasiveness in Everyday Conversations
- Evaluating the Capabilities of LLMs for Persuasive Dialogue
- Large Language Models are as persuasive as humans, but how? About the cognitive effort and moral-emotional language of LLM arguments
- Exploring the Role of Prior Beliefs for Argument Persuasion
- When Large Language Models are More Persuasive Than Incentivized Humans, and Why
- The Thin Line Between Comprehension and Persuasion in LLMs