INQUIRING LINE

Does slick writing make us stop checking if a claim is actually true?

Does fluency and polish in presentation substitute for truth evaluation in readers?

This explores whether a smooth, confident, well-formatted presentation leads people (and AI judges) to accept a claim as true without actually checking it, and what the corpus says about why that happens and whether anything breaks the effect.


This explores whether polish stands in for the work of checking truth. The corpus's short answer is yes, often, and more completely than most people would guess. In an 81-person study, readers given no signal about where claims came from showed no measurable ability to tell fluent fabrications from ground truth. Their accuracy was at chance. They regained a clear gap only when an interface showed them which claims were backed by verified evidence Can readers tell truth from fabrication without evidence signals?. So the reader hasn't simply lowered their standards. Without outside cues, there's nothing left in the text for them to judge by except how it reads.

The effect goes beyond 'believing the text.' Evaluators shown AI-generated documents judged them both human-written and better than real human submissions Does polished writing actually signal better quality work?. Smaller models trained to imitate ChatGPT's confident style fooled human raters into thinking they had improved, even though their factual accuracy hadn't moved Can imitating ChatGPT fool evaluators into thinking models improved?. Machines aren't immune either. LLM judges fall for fake references and rich formatting, and these tricks work no matter what the content says Can LLM judges be fooled by fake credentials and formatting?. Polish works on both human and machine evaluators.

Part of the explanation is about how fluency is produced, not just how it's received. LLMs make about 77% fewer of the small 'grounding' moves that humans use to check understanding, like clarifying questions and acknowledgments. Preference training actively removes these moves because raters reward confident, complete-sounding answers Why do language models sound fluent without grounding?. The smoothness partly comes from skipping the checking work. The interface adds to this: confident tone, clean grammar and tidy formatting create a halo effect, and chat interfaces nudge users toward the next question rather than back to the source Does confident AI tone make users skip fact-checking?. Linguistics finds a related mechanism. Claims slipped in as background assumptions persuade better than stated claims because they never get presented for evaluation at all Why are presuppositions more persuasive than direct assertions?.

The less obvious finding is that fluency misleads people about themselves as well as about the text. Users who work with polished AI output start to feel more competent, as if the smoothness they experience reflects their own understanding Does processing ease mislead users about their own competence?. Writers who use AI assistance are read by others as more confident, more extreme and higher-quality, across all 29 traits measured Does AI writing assistance change how readers perceive the writer?. Polish changes how readers judge the claim, the author and their own grasp of the material.

Two caveats keep this from being the whole story. In debates, what readers already believe predicts what persuades them better than any linguistic feature does Does what readers believe matter more than what debaters say?. Fluency seems to matter most when readers have no strong prior and no outside evidence. That's also where the fix lies. Since readers can't extract truth from the text alone, the promising remedy is to show them evidence and provenance, not to make the prose sound less sure of itself Can readers tell truth from fabrication without evidence signals?.


Sources 10 notes

Can readers tell truth from fabrication without evidence signals?

In an 81-person study, participants given no provenance cues showed no significant truth discernment (p = .43), falling for fluent hallucinations as readily as ground truth. An idealized Provenance Density interface showing verified claims restored a +4.15 point gap (p < .001).

Does polished writing actually signal better quality work?

Studies show evaluators perceived AI-generated documents as both human-written and better quality than human submissions. This suggests rhetorical polish misleads judgment and should not serve as a quality signal in evaluation.

Can imitating ChatGPT fool evaluators into thinking models improved?

Imitation models fool human evaluators by mimicking ChatGPT's confident, fluent style while failing to improve factuality or generalization on novel tasks. The ceiling is set by base model capability, not fine-tuning method—better fundamentals, not shortcuts, drive real improvement.

Can LLM judges be fooled by fake credentials and formatting?

Research identified four evaluation biases in LLM judges, with authority and beauty biases being semantics-agnostic and trivially exploitable through fake references and formatting—zero-shot attacks requiring no model access or optimization.

Why do language models sound fluent without grounding?

LLMs generate 77.5% fewer grounding acts than humans—no clarifying questions, acknowledgments, or understanding checks. Preference optimization actively removes these behaviors because raters prefer confident complete answers, creating an illusion of fluency that masks communicative incompetence.

Show all 10 sources
Does confident AI tone make users skip fact-checking?

Samsonov argues that AI's confident tone, correct grammar, and polished formatting create a halo effect that makes users trust accuracy without verification. The chat interface compounds this by offering no accuracy warnings and actively prompting users toward new questions instead of source-checking.

Why are presuppositions more persuasive than direct assertions?

Experimental evidence shows presuppositions with additive, iterative, and factive triggers persuade audiences more than assertions, especially for discourse-new content. The mechanism: presuppositions bypass evaluative scrutiny by presenting claims as already-accepted background.

Does processing ease mislead users about their own competence?

High-quality AI output triggers a metacognitive heuristic: users experience fluency as a signal of their own capability, even though they didn't generate it. This self-directed fluency illusion systematically inflates perceived competence because LLMs optimize for fluency regardless of user understanding.

Does AI writing assistance change how readers perceive the writer?

A study of 2,939 writers and 11,091 readers found AI assistance shifted every tested dimension—29 total—toward extremism, confidence, quality, agreeableness, and perceived privilege. Distortions were statistically significant and directional, not random noise.

Does what readers believe matter more than what debaters say?

Analysis of debate corpora shows that political and religious ideology labels of voters outpredict linguistic features when modeling debate outcomes. Language effects observed without reader controls are confounded by audience composition correlated with debate topics.

Papers this line draws on 8

The research behind the notes this line reads — ranked by how closely each paper relates.