Do conversational agents make it harder to spot false information?

Anderl, Klein, Sarigül, Schneider, Han, Fiedler and Utz report two preregistered experiments testing whether presentation mode changes how well people catch inaccurate information. Participants rated the credibility of accurate versus partially inaccurate answers to six general-knowledge questions, presented either as static text (a Wikipedia-style encyclopedia), dynamic text (a ChatGPT-style agent), or voice (an Alexa-style agent), with the information content held constant across conditions. Experiment 1 (N = 556 after exclusions) found participants "were better at detecting inaccuracies when identical information was provided as static text compared to both types of conversational agents." Experiment 2 repeated the test with unbranded versions of the same three modes and found the pattern held independent of whether the application was named ChatGPT, Alexa, and Wikipedia or left generic, which the authors read as evidence the effect is about conversational form, not brand reputation.

The authors' account runs through two mechanisms acting in parallel: a modality effect (reading versus listening) and a conversational-nature effect (dynamic/interactive versus static presentation). Reading aids discernment relative to listening, while conversational framing degrades it relative to non-conversational framing. Because the dynamic text-based agent combines reading (which helps) with conversational framing (which hurts), it lands between the voice-based agent (listening and conversational, both hurting discernment) and static text (reading and non-conversational, both helping). Mediation analysis points to processing fluency and perceived enjoyment as factors that, when compared between the two conversational agents, "seemed to partially (Experiment 1) or fully (Experiment 2) suppress the effect of presentation mode on credibility" — and notably, the pattern ran opposite to participants' own self-reported trust ratings of the applications, meaning people's stated trust in ChatGPT, Alexa, or Wikipedia did not predict which mode left them better able to spot false information.

This gives experimental, information-content-controlled backing to Does conversational style actually make AI more trustworthy?, whose focus-group participants reported trusting ChatGPT for its contingency and directness rather than its accuracy; this study shows that same conversational quality measurably reduces discernment even when the underlying information is identical across modes, and the paper itself contrasts its controlled design with "previous research showing similar findings for actual information search that were however observed in a much less controlled setting." It also parallels Can readers tell truth from fabrication without evidence signals?, where polish and fluency stood in for truth in reader judgments of prose; here conversational style and listening (not textual polish) are the surface cues that substitute for accuracy-checking, a different route to the same discernment failure. And it sits alongside Does access to web search prevent overreliance on chatbots?, since both studies separate what predicts belief (conversational form, prior trust) from what should predict it (actual accuracy).

The excerpt is explicit that its mediation analysis is correlational: the proposed mediators (processing fluency, social presence, enjoyment) were not independently manipulated, so the paper "does not allow for the conclusion that they are indeed the causal drivers of the observed effects." The authors also flag that real conversational agents, with more interactivity and familiarity than their fixed six-snippet stimuli, might show a stronger effect, not a weaker one. The finding that conversational presentation mode lowers inaccuracy detection is drawn from controlled, preregistered experiments; what still needs testing is whether the same gap holds in live, multi-turn use rather than single-exchange snippets.