INQUIRING LINE

Use AI to write or code, and you may start believing you're better at it than you are.

How does AI assistance change people's perception of their own competence?

This explores how using AI changes the way people judge their own skills, whether they think they got better, and whether that judgment matches reality.


This explores how using AI changes the way people judge their own skills. The corpus points to one main effect: people credit themselves for what the AI did. Researchers call it the LLM Fallacy, a self-perception error where AI-assisted output gets absorbed into a person's sense of what they can do (Do AI-assisted outputs fool users about their own skills?). It is not hallucination or automation bias. It happens regardless of whether the output is accurate or whether the person over-relies on it (How does AI-assisted work reshape how people see their own abilities?). Making the AI more accurate won't fix it, because the problem is in how people see their own contribution.

Several things feed the illusion. Four mechanisms interact: it's unclear who did what, fluent output feels like skill, thinking gets handed off to the tool, and the pipeline behind the output is hidden. The effect is multiplicative, so each mechanism amplifies the others (How do AI tools trick users into overestimating their own skills?). The fluency piece is the most striking. People read the ease of a polished result as a sign of their own ability, and LLMs produce polish whether or not the user understood anything (Does processing ease mislead users about their own competence?). The illusion is strongest when the output is seamless and the human-AI boundary disappears.

Asking people how competent they feel won't catch this. A pooled analysis of three studies found a correlation of just .055 between self-reported and objectively measured AI competence, with confidence intervals that include zero (Can self-ratings replace objective performance scores for AI competence?). Real ability may also be moving the other way. In a four-month EEG study, heavier LLM use went with weaker brain connectivity, worse memory, and difficulty recalling one's own recent work (Does AI assistance weaken our brain's ability to think independently?). Even correct AI suggestions have a cost: they break the immersion that good reasoning depends on (Does AI assistance always help reasoning or does it carry hidden costs?). Feeling more capable while being measurably less engaged is a hard gap to see from the inside.

The same fluency trick works on other people too. In a study of 2,939 writers and 11,091 readers, AI assistance shifted how readers perceived the writer on all 29 dimensions tested, toward more confident, higher quality, and more agreeable (Does AI writing assistance change how readers perceive the writer?). Judges of AI models fall for it as well. Models trained to imitate ChatGPT fooled human evaluators with a confident style while closing no real capability gap (Can imitating ChatGPT fool evaluators into thinking models improved?). In every case, polish gets read as competence. The model can't reliably flag where its contribution ends either, since its self-reports are unstable (How well do language models understand their own knowledge?). And people judge AI partners mainly on perceived competence, which accounts for about half of the variance in their impressions (How do users mentally model dialogue agent partners?).

The unexpected takeaway is that a more seamless AI makes the misperception worse, because the seamlessness is what hides who did the work. The fixes the corpus suggests are about making the boundary between human and machine contribution visible, not about making the AI more correct.


Sources 11 notes

Do AI-assisted outputs fool users about their own skills?

Research identifies a systematic cognitive attribution error where individuals integrate AI-generated outputs into their capability identity, believing they possess skills they don't actually have. This occurs when task output is seamless and fluent, obscuring the human-AI boundary.

How does AI-assisted work reshape how people see their own abilities?

Research shows the LLM Fallacy operates through misattribution of AI outputs to personal capability, independent of output accuracy or reliance behavior. It requires interventions that clarify human-machine contribution boundaries, not just better system accuracy or forced verification.

How do AI tools trick users into overestimating their own skills?

Attribution ambiguity, fluency illusion, cognitive outsourcing, and pipeline opacity combine to systematically misattribute AI outputs as user competence. The effect is multiplicative—each mechanism amplifies the others.

Does processing ease mislead users about their own competence?

High-quality AI output triggers a metacognitive heuristic: users experience fluency as a signal of their own capability, even though they didn't generate it. This self-directed fluency illusion systematically inflates perceived competence because LLMs optimize for fluency regardless of user understanding.

Can self-ratings replace objective performance scores for AI competence?

A pooled analysis of three studies found a correlation of only .055 between self-reported and objective measures of AI competence, with confidence intervals including zero. This provides no basis for substituting self-assessment for demonstrated performance.

Show all 11 sources
Does AI assistance weaken our brain's ability to think independently?

A four-month EEG study of 54 participants found that brain connectivity systematically scaled down with AI reliance—LLM users showed weakest neural engagement, poorest memory retention, and impaired ability to recall their own recent work.

Does AI assistance always help reasoning or does it carry hidden costs?

Well-intentioned AI suggestions can damage reasoning performance by severing cognitive immersion, forcing users to rebuild focus before continuing. Evaluation must measure flow preservation across entire tasks, not just local suggestion accuracy.

Does AI writing assistance change how readers perceive the writer?

A study of 2,939 writers and 11,091 readers found AI assistance shifted every tested dimension—29 total—toward extremism, confidence, quality, agreeableness, and perceived privilege. Distortions were statistically significant and directional, not random noise.

Can imitating ChatGPT fool evaluators into thinking models improved?

Imitation models fool human evaluators by mimicking ChatGPT's confident, fluent style while failing to improve factuality or generalization on novel tasks. The ceiling is set by base model capability, not fine-tuning method—better fundamentals, not shortcuts, drive real improvement.

How well do language models understand their own knowledge?

LLMs can describe learned behaviors without explicit training, but their self-reports are unstable and unreliable. Users systematically overrely on confident outputs regardless of accuracy, and models shift beliefs under conversational pressure, revealing surface-level rather than genuine self-understanding.

How do users mentally model dialogue agent partners?

The Partner Modelling Questionnaire reveals that perceived competence dominates user impressions (49% of variance), followed by human-likeness (32%) and communicative flexibility (19%). This three-factor structure reflects how people evaluate dialogue partners against both functional and social standards.

Papers this line draws on 8

The research behind the notes this line reads — ranked by how closely each paper relates.