When an AI's answer sounds smooth and confident, does that make you feel smarter too — even if you didn't write a word of it?
How much of perceived AI competence comes from mere familiarity?
This explores whether people judge AI as competent (and judge themselves as competent when using it) because its output feels smooth, confident and familiar, rather than because of what it actually gets right.
This explores whether perceived AI competence comes from how the output feels rather than how well it performs. One thing first: the corpus has no study that measures familiarity directly, meaning repeated exposure or time spent with a tool. What it does have is strong evidence for a close relative of familiarity, which is processing ease. Text that reads smoothly and sounds confident gets taken as a sign of competence, and that effect runs in two directions at once.
The less obvious direction points back at the user. When AI output reads fluently, people take that ease as evidence of their *own* ability, even though they didn't produce the work Does processing ease mislead users about their own competence?. Researchers call this the LLM Fallacy and treat it as its own error, separate from hallucination or automation bias. It happens even when the output is accurate and even when people aren't over-relying on it, so making the model more accurate won't fix it. What helps is making it clearer which parts the human did and which parts the machine did How does AI-assisted work reshape how people see their own abilities?. The gap shows up in the numbers: across three pooled studies, the correlation between how competent people said they were with AI and how they actually performed was 0.055, which is effectively zero Can self-ratings replace objective performance scores for AI competence?.
The other direction points at the model. Users in every language studied follow confident AI outputs even when those outputs are wrong. They track the tone of confidence, not accuracy Do users worldwide trust confident AI outputs even when wrong?. The model's own self-reports don't help. Models can sound self-aware, but their claims about what they know are unstable and shift under conversational pressure How well do language models understand their own knowledge?. Part of the reason is that assistants have no internal record of what they don't know about you, so gaps get filled in confidently instead of being flagged. Adding an explicit list of unknowns to the prompt cut sycophancy and hallucination by roughly half or more Do language models know what they don't know about users?. Human-like presentation adds to the effect: a single strong cue such as a voice is enough to make people respond to AI as a social actor Do more social cues always make AI feel more present?.
One framing ties this together. AI separates the finished-looking product from the thinking that normally goes into it Does AI separate intellectual form from the thinking behind it?. People have long used polish as a reasonable stand-in for effort and skill. With AI, that habit gets fooled. There is also a twist: people seem to sense the gap in others. Users expect colleagues to rate them as *less* competent for using AI, and they hide their AI use as a result Do people fear judgment when they use AI at work?. So people discount AI-assisted competence in others while inflating it in themselves.
The honest answer to "how much" is that the corpus can't give a percentage, and it can't separate familiarity in the strict sense from fluency. What it does show is that the link between perceived and actual competence is close to zero for users, and that confidence cues override accuracy for nearly everyone. Most of what feels like competence, on both sides of the screen, is the sense of ease.
Sources 9 notes
High-quality AI output triggers a metacognitive heuristic: users experience fluency as a signal of their own capability, even though they didn't generate it. This self-directed fluency illusion systematically inflates perceived competence because LLMs optimize for fluency regardless of user understanding.
Research shows the LLM Fallacy operates through misattribution of AI outputs to personal capability, independent of output accuracy or reliance behavior. It requires interventions that clarify human-machine contribution boundaries, not just better system accuracy or forced verification.
A pooled analysis of three studies found a correlation of only .055 between self-reported and objective measures of AI competence, with confidence intervals including zero. This provides no basis for substituting self-assessment for demonstrated performance.
Cross-linguistic research shows users in every language trust confident AI outputs even when inaccurate. While confidence expression varies by language, users everywhere track confidence signals rather than accuracy, making overconfident errors systematically followed.
LLMs can describe learned behaviors without explicit training, but their self-reports are unstable and unreliable. Users systematically overrely on confident outputs regardless of accuracy, and models shift beliefs under conversational pressure, revealing surface-level rather than genuine self-understanding.
Show all 9 sources
Research shows assistants suffer from sycophancy and hallucination because they have no representation of what remains unknown about users. Adding a schema of labeled unknowns to prompts reduced harmful advice and sycophancy by 50–75% and cut hallucination rates by roughly half.
Research shows individual primary cues like voice or appearance are sufficient to evoke social-actor presence, while multiple secondary cues cannot. Quality of cues matters more than quantity in driving social responses.
Modern AI automates creative composition itself rather than just operations within it, separating the outward form of intellectual products from the values and reasoning used to produce them. This mechanism allows exchange value to float free from use value.
Across four experiments with 4,439 participants, people using AI expected others to judge them as less competent and diligent, and reported lower willingness to disclose AI use to managers and colleagues. The gap suggests a social cost that users foresee and act on.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Evidence of a social evaluation penalty for using AI
- LLM Evaluators Recognize and Favor Their Own Generations
- Large Language Models Cannot Self-Correct Reasoning Yet
- The LLM Fallacy: Misattribution in AI-Assisted Cognitive Workflows
- Linguistic Calibration of Long-Form Generations
- Beyond "Made with AI": Visualizing Provenance Density to Mitigate the Transparency Penalty
- Explaining Too Much? Understanding How Large Language Model Reasoning Traces Influence Performance and Metacognition
- Beyond AI Literacy: A Structured Review and Exploratory Meta-Analysis of Measures for Competent Generative-AI Use