People often rate AI moral advice as sharper than human advice — until they learn an AI actually said it.
Do people judge AI moral advice differently than human advice when they know the source?
This explores whether people judge moral advice differently once they learn it came from an AI rather than a person, and whether that label changes what they actually do with the advice.
This explores whether learning that moral advice came from an AI changes how people judge it and act on it. The corpus gives a split answer, and the split is the interesting part. In blind comparisons, people often rate AI moral reasoning higher than human reasoning, especially in complex scenarios. Once they're told the argument came from an AI, their agreement drops. The researchers suggest that liking the content and rejecting the source come from separate psychological processes, so a person can admire an argument and still distrust where it came from Do people prefer AI moral reasoning when they don't know the source?.
A second line of work points the other way. When researchers measured whether people actually followed moral advice, it persuaded them about equally whether it was labeled as coming from an LLM or a human expert. The advisor's track record made little difference. So did the quality of its reasoning: good reasons didn't add persuasive force, though bad reasons took some away. People seemed to defer to the bottom-line recommendation as a shortcut rather than weighing the argument Does LLM moral advice persuade through reasoning or just recommendations?. Put the two findings together and the AI label seems to change what people say they agree with more than what they end up doing. A related finding goes beyond advice: dishonest AI peers pushed people toward dishonesty about as much as dishonest human peers did Do AI peers influence human dishonesty like human peers do?. Social influence doesn't seem to check whether a human is behind it.
The source question only matters if people can tell the sources apart, and often they can't. In medicine, participants identified AI answers versus doctors' answers at chance level. They also trusted low-accuracy AI answers enough to act on them Can people tell AI medical advice from doctors' responses?. Even clinicians couldn't reliably tell GPT-4's advice from expert advice, and they rated it higher on emotional empathy Can clinicians tell GPT-4 advice apart from expert advice?. One possible reason AI moral text reads so well is that LLMs use about 22% more moral language than humans (appeals to care, fairness, authority and sanctity) while keeping the same emotional tone Do LLMs use moral language more than humans?. These studies didn't directly measure whether that extra moral language drives the blind-test preference, so treat the link as a plausible guess.
The more philosophical notes ask whether the drop in trust after disclosure is just bias, or whether it tracks something real. One argument holds that expert judgment involves more than producing correct content. Experts anticipate how their audience will receive their advice and stand behind it socially. On this view, AI's fluent advice can mislead because it has the form of expertise without that accountability Can AI replicate the communicative work experts do?. An anthropological reading makes a similar point: advice from a person carries a relationship and an obligation, while AI output was never anyone's to give Why doesn't AI output carry the spirit of a giver?. From that angle, discounting AI moral advice once you know its source may be a reasonable response to the fact that nobody stands behind it.
If you want a practical framework, one proposal says to treat AI output as one input to your own reasoning rather than a replacement for it. You would set it aside when bias, a domain mismatch or new evidence shows up Should AI outputs replace or supplement human judgment?. Another design has the AI point out what matters in a decision instead of handing over a verdict, which avoids the anchoring problem that recommendation-first advice creates Can AI guidance reduce anchoring bias better than AI decisions?. Given the deference finding above, that design choice may matter more for moral advice than any disclosure label.
Sources 10 notes
Participants rated utilitarian moral arguments higher when attributed to LLMs, but agreement dropped when told the arguments were AI-generated. The preference for content and rejection of source operate independently through different psychological processes.
Three studies show LLM moral advice persuades equally whether attributed to AI or human experts, regardless of track record or reasoning quality. Good reasons don't increase persuasion; bad reasons weaken it—suggesting people defer to recommendations as peripheral cues rather than evaluating arguments.
In two randomized experiments, participants reported more dishonestly when exposed to dishonest AI peers compared to honest ones, with effect sizes comparable to human peer influence. The effect held across different norm conditions but showed diminishing returns with more dishonest peers.
A 300-participant study found participants could not reliably distinguish AI-generated medical responses from doctors' (50% accuracy, chance level) and rated low-accuracy AI answers as valid and trustworthy enough to act on them—comparable to or stronger than their trust in actual doctors' advice.
Blinded clinician ratings of 104 response pairs found GPT-4 advice favored on emotional empathy, with no significant differences in scientific quality or cognitive empathy. Clinicians identified the source at chance level (45% accuracy), suggesting the two were indistinguishable in written form.
Show all 10 sources
Research comparing LLM and human arguments found that LLMs used significantly more moral framing across care, fairness, authority, and sanctity foundations, despite producing sentiment scores nearly identical to humans. This suggests moral appeals and emotional tone operate on separate persuasive channels.
Expertise requires anticipating audience acceptability and social validity, not just retrieving information. AI lacks the mechanism to perform this communicative work, making its fluent output epistemically misleading despite its confident form.
AI-generated content lacks hau—the spiritual essence that binds gift economies—because no person gave it. This absence is more fundamental than alienation: the output was never anyone's to begin with, so no relationship of obligation forms.
Research argues AI should supplement rather than replace human reasoning, with deference withdrawn when domain mismatch, bias, conflicting authority, or new evidence emerges. This prevents opacity-driven failures that full preemption would mask.
Learning to Guide eliminates anchoring bias and unassisted hard cases by having machines supply interpretive guidance rather than autonomous decisions, keeping responsibility with humans while improving their judgment through enhanced perception.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- People Defer to AI Moral Advice, But Not Blindly
- People Overtrust AI-Generated Medical Advice despite Low Accuracy
- "That's AI Slop, You Bot!" Studying Accusations, Evidence, and Credibility in Online Discourse Towards LLM-Generated Comments
- The Moral Turing Test: Evaluating Human-LLM Alignment in Moral Decision-Making
- Large Language Models are as persuasive as humans, but how? About the cognitive effort and moral-emotional language of LLM arguments
- Incoherent by Design? On the Moral Self-Consistency of LLMs
- Large Language Models Do Not Simulate Human Psychology
- GenAI as a Power Persuader: How Professionals Get Persuasion Bombed When They Attempt to Validate LLMs