When an AI fact-checker gets it wrong, do people end up believing worse things than if nobody checked at all?

This explores whether AI fact-checkers that get things wrong can leave people worse off than no fact-check at all. The short answer from the corpus is yes. The harm is lopsided: the errors move beliefs more than the correct verdicts fix them. A randomized trial found AI fact-checking did not improve people's overall ability to tell true headlines from false ones Does AI fact-checking actually help people spot misinformation?. The failures pulled in two directions. When the AI wrongly flagged a true headline as false, people believed it less. When the AI said it was unsure about a false headline, people believed it more. Hedging, the move that sounds most responsible, ended up working as a quiet endorsement of misinformation. The finding with the sharpest edge is about who suffers: people who chose to use the fact-checker shared more content and believed more misinformation. The tool may draw in the very users it misleads most.

Why would a wrong label carry so much weight? Part of the answer is the delivery format. People are measurably worse at spotting inaccurate answers when those answers come from a chatbot or voice assistant than when the same information appears as static, Wikipedia-style text, and this holds regardless of branding Do conversational agents make it harder to spot false information?. Fluent, conversational output lowers our guard. Samsonov makes a related point about the interface itself: confident tone, clean grammar and polished formatting create a halo effect, and chat interfaces nudge users toward the next question rather than toward checking sources Does confident AI tone make users skip fact-checking?. A fact-check is supposed to be the checking step. When it arrives in a format that discourages checking, a mistake gets little scrutiny.

The errors are also not rare. Across 1.67 million fact-checks on X, Grok and Perplexity bots agreed with human fact-checkers only 54–58% of the time, below the 64% rate at which human fact-checkers agree with each other Do AI fact-checkers on social media match human accuracy?. One surprise: Grok accessed through the API matched human consensus. So the gap seems to come from how the consumer bots are deployed, not from what the model can do, which means the mislabeling problem can at least partly be fixed. A second surprise from the same researchers: simply naming Grok as the source made Democrats update their beliefs less and Republicans update more, while naming Perplexity made no partisan difference Does naming Grok as fact-checker change how people update beliefs?. The same verdict can move people in opposite directions depending on the brand attached to it. That means 'net harm' has to be measured separately for each audience, not as one average.

One adjacent finding complicates the hope that people will push back. In a BCG study of consultants, challenging GPT-4's output often made the model argue harder rather than correct itself Does validating AI output make models more defensive?. A user who doubts a wrong fact-check may get a more persuasive version of the same error. Training choices can make this worse: models tuned to sound warm become less truthful and worse at resisting disinformation, especially when users voice false beliefs Does empathy training make AI systems less reliable?.

The takeaway you might not have expected: the main risk is not that AI fact-checkers are sometimes wrong, since human fact-checkers disagree too. The risk is that the AI's mistakes and its hedges carry more persuasive weight than its correct calls, and they reach people through a channel that weakens their scrutiny. The corpus has strong evidence on headline belief, but little on long-term effects or on how often AI fact-checks should be corrected or retracted. Those questions remain open.