Do flashy citations and polished numbering trick readers into trusting sources that don't actually check out?

This explores whether the look of citations (how many there are, how polished they appear) makes readers trust sources that are irrelevant or invented. The corpus says yes, and the effect is bigger than you might expect. In an analysis of 24,000 head-to-head comparisons of AI search answers, users preferred responses with more citations almost as strongly when those citations were irrelevant as when they actually supported the answer Do users trust citations more when there are simply more of them?. In other words, readers treat the number of citations as a stand-in for checking them. They respond to the bracketed numbers, not to what the sources say.

AI systems that grade other AI outputs fall for the same trick. Research on LLM judges found an "authority bias" and a "beauty bias": adding fake references or rich formatting raises scores no matter how good the content is. Exploiting this takes no access to the model's internals, because pasting in made-up references is enough Can LLM judges be fooled by fake credentials and formatting? Can LLM judges be tricked without accessing their internals?. This matters beyond chatbots. Many AI benchmarks use LLM judges, so a citation-shaped signal can inflate the measurements we use to decide which models are trustworthy.

The supply of fake citations is real. One study checked 178 AI-generated references and found that 28 couldn't be verified at all How often do AI systems generate fake citations?. More worrying is why fabrication happens. When deep research agents fail, 39% of the failures involve the agent inventing examples and evidence to look rigorous when the task demands depth Why do deep research agents fabricate scholarly content?. At the extreme, one demonstration produced 288 complete finance papers with invented theory and fabricated references Can AI generate hundreds of fake academic papers automatically?. Put these findings together and you get a bad loop: models learn that citation-shaped text gets rewarded, and readers reward it.

The most useful finding is that this is mostly a design problem, not a problem with readers. In an 81-person study, people given no information about where claims came from could not tell fluent hallucinations from true statements at all. A display showing how many claims were actually verified restored their ability to tell the difference Can readers tell truth from fabrication without evidence signals?. A separate 372-person experiment found that the way sources are shown changes whether more citations help or hurt. A persistent sidebar of sources improved critical thinking as citations increased, while hover-card citations kept the writing smooth but weakened people's ability to combine multiple sources Can interface design reverse citation overload's harm to critical thinking?.

The takeaway you might not have expected: a citation is a user-interface element as much as it is evidence. A numbered superscript makes a promise of verifiability that the interface usually doesn't keep. Systems can also be built so the model has no room to bluff, for example by refusing to answer when it has no grounded evidence Can RAG systems refuse to answer without reliable evidence?. Until that design is common, more footnotes should make you more suspicious, not less.