Do users trust citations more when there are simply more of them?
Explores whether citation quantity alone influences user trust in search-augmented LLM responses, independent of whether those citations actually support the claims being made.
Search Arena provides the largest analysis of user preferences for search-augmented LLMs: over 24,000 paired multi-turn interactions with ~12,000 human preference votes. The finding that matters most: users prefer responses with more cited sources, and this preference extends to irrelevant citations.
The effect sizes are nearly identical. Correctly attributed citations have a positive coefficient of β=0.285 on user preference. Irrelevant citations — citations that do not support the associated claims — have a positive coefficient of β=0.273. Users are influenced by the presence of citations roughly equally regardless of whether those citations actually back up the text.
This means citation count functions as a surface trust heuristic, decoupled from citation quality. Users see citations and infer credibility without verifying the cited content supports the claim. The gap between perceived and actual credibility is systematic, not incidental.
Additional preference signals: users prefer community-driven platforms (tech blogs, social networks) over encyclopedic sources like Wikipedia. Reasoning-enhanced responses are preferred. Longer responses are preferred. Web search does not degrade and may improve performance in non-search settings — but search settings are significantly affected when relying solely on parametric knowledge.
This connects to Do users worldwide trust confident AI outputs even when wrong?. In that finding, confidence signals override accuracy assessment. Here, citation signals override quality assessment. Both are instances of the same pattern: users use surface proxies for quality because evaluating actual quality is cognitively expensive.
The implication for RAG system design is direct: optimizing for user satisfaction and optimizing for answer quality are not the same optimization target. A system can score highly on user preference by adding more citations — even irrelevant ones — without improving answer quality. This is a form of metric gaming at the human-evaluation level.
Inquiring lines that read this note 82
This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.
How do evaluation biases undermine LLM quality assessment systems?- How do LLMs generate false citations that sound like real scholarship?
- Why do some LLM clusters cite broader psychology than others?
- Does exposure to more domain-specific examples reduce LLM overconfidence?
- Why does probability of text completion not equal knowledge value?
- Why does automated evaluation consistently overestimate research quality?
- Can crowdsourced voting and automated panels both credibly evaluate LLM outputs?
- Can statistical filtering plus narrative generation fool academic peer review?
- Does complexity signal credibility and authority to readers?
- Why do citation counts increase trust even without relevance?
- How do retrieval failures enable generation of fabricated scholarly constructs?
- How do experts select which other experts to trust?
- Does high knowledge density in text reduce user motivation to read more?
- Can factual product data improve the credibility of subjective opinion summaries?
- Why do users prefer community sources over encyclopedic references?
- How do citation patterns encode collective judgment about research quality?
- Why are documents read but not cited harder distractors than random samples?
- Why do readers trust citations more even when they are irrelevant?
- Does provenance alone guarantee that cited sources are actually sound?
- Why do users trust citations even when they are irrelevant?
- How much does citation grounding help if agents ignore the citations?
- How does social proof work differently when there is no identifiable author?
- Does stripping social context from knowledge claims hollow out their meaning?
- How does AI fact-checking compare to other trust signals like citation counts?
- What role should the trust parameter play in using synthetic data as evidence?
- What role does commitment and reputation play in building trustworthy expertise?
- Can citation practices work when AI cannot produce traceable sources?
- What prevents scholarly infrastructure from filtering out ghost-authored records automatically?
- Can beam search and ranking functions evaluate claims without understanding counterarguments?
- Why does evaluating multiple candidates work better than judging one answer?
- Can persuasion effects that avoid demographic profiling maintain factual accuracy?
- How does source attribution change the complexity-persuasion relationship?
- How does collapsing the author-public distinction remove the audience an appeal would target?
- How does social standing give certain claims more persuasive power than others?
- Why do aggregate persuasion metrics mask what actually changes minds?
- How does persuasive framing replace evidence in contested domains?
- Does expressed certainty actually persuade users more than evidence?
- Why do LLM explanations cite similarity and diversity more as options increase?
- Does post-hoc justification increase when LLM choices become harder to defend?
- How does explanation fluency mislead users about actual recommendation procedures?
- How does pretraining corpus popularity bias affect LLM recommendation behavior?
- Can confidence levels improve recommendations compared to single-number ratings?
- What conversational moves signal expertise and build credibility in recommendations?
- Does uncertainty quantification in model responses reduce persuasive impact on audiences?
- How do one-sided explanations act as confidence signals to users?
- Do verbal uncertainty estimates calibrate better than confidence scores for personalization?
- How does confidence in LLM outputs override users' ability to check accuracy?
- How does understanding persistent journeys intensify both trust and privacy concerns?
- How does personalization increase trust while degrading clinical safety outcomes?
- Does the interface design itself shape how much content users will review?
- What anchoring effects shape how users rate items in sequence?
- How much do social audience effects distort the true average satisfaction in review aggregates?
- Does weak versus robust anthropomimesis produce different user trust responses?
- Can anonymity and trustworthiness coexist in online spaces without credential systems?
- How does search budget affect answer quality at test time?
- How do real search queries reveal what counts as a deep research question?
- What makes search budget matter for research task performance?
- Can graded relevance assumptions hold when user ratings are temporally inconsistent?
- What documents improve answers beyond surface query similarity?
- Can adaptive elbow detection replace fixed top-k limits in evidence retrieval?
- Why does adaptive document allocation improve over fixed k selection?
- Can RAG systems game user preferences by adding irrelevant citations?
- What role does document reranking play alongside decisions about whether to retrieve?
- Why do current benchmarks fail to match user satisfaction with search results?
- What role does vague intent play in realistic search evaluation?
Related concepts in this collection 3
This note in its neighbourhood — explore the map, then jump to a related concept in the list below.
Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph
-
Do users worldwide trust confident AI outputs even when wrong?
Explores whether the tendency to over-rely on confident language model outputs transcends language and culture. Understanding this pattern is critical for designing safer human-AI interaction across diverse linguistic contexts.
same pattern: surface signals override quality evaluation
-
Can LLM judges be fooled by fake credentials and formatting?
Explores whether language models evaluating text fall for authority signals and visual presentation unrelated to actual content quality, and whether these weaknesses can be exploited without deep model knowledge.
citation inflation is another bias axis exploitable in evaluation systems
-
Can LLM explanations actually help humans predict model behavior?
Do model explanations enable users to accurately simulate how the model will behave on related inputs? This matters because it determines whether explanations genuinely improve human understanding or just create an illusion of understanding.
plausibility ≠ precision mirrors citation-count ≠ citation-quality
Related papers in this collection 8
Papers most semantically related to this note, ranked by cosine similarity in the embedding space.
- Search Arena: Analyzing Search-Augmented LLMs
- Flattery, Fluff, and Fog: Diagnosing and Mitigating Idiosyncratic Biases in Preference Models
- LLMs can be Fooled into Labelling a Document as Relevant
- AI Can Learn Scientific Taste
- Query Understanding in the Age of Large Language Models
- News Source Citing Patterns in AI Search Systems
- Think Twice Before Trusting: Self-Detection for Large Language Models through Comprehensive Answer Reflection
- From speaking like a person to being personal: The effects of personalized, regular interactions with conversational agents
Original note title
users prefer responses with more citations even when citations are irrelevant — citation count is a decoupled trust heuristic