People's own sense of their AI skill barely predicts how well they actually use it — so what should replace self-assessment?
How do self-reported skill outcomes compare to mode-driven predictions?
This explores how well people's own reports of their AI skill line up with what actually happens, compared with predictions drawn from data such as conversation logs or a model's own track record. I'm reading 'mode-driven' as 'model-driven'.
This explores how well people's own reports of their AI skill line up with what actually happens, compared with predictions drawn from data such as conversation logs or a model's own track record. (I'm reading 'mode-driven' as 'model-driven'.) The corpus gives a fairly blunt answer. Self-reports barely predict anything, and data-driven signals do better, though they come with their own catch.
Start with self-reports. A pooled analysis of three studies found a correlation of about .055 between how competent people said they were with generative AI and how they actually performed. The confidence interval included zero Can self-ratings replace objective performance scores for AI competence?. In practice, knowing someone's self-rating tells you almost nothing about their results. That matters because many workplace AI-skills surveys and training evaluations rely on exactly this kind of self-assessment.
The data-driven side looks better. When researchers used machine learning to pull interpretable traits from people's conversations with coding agents, those traits explained coding outcomes beyond what prior achievement predicted Can conversation patterns predict coding outcomes better than prior skill?. How you actually talk to the AI says more than how good you think you are. The catch is that these traits weren't stable or transferable enough to count as a learnable 'collaboration skill.' They predict outcomes without telling you what to teach. So the comparison isn't a clean win for either side. Self-reports fail to predict, and behavioral signals predict without explaining.
The same pattern shows up when the 'self' doing the reporting is the model. LLM self-reports mostly echo how humans talk about themselves in the training data rather than reflecting real internal states. Real introspection appears only when there's a causal link between the internal state and the report Can language models actually introspect about their own states?. The practical fix mirrors the human case. XConf gets better confidence estimates by ignoring the model's in-the-moment sense of certainty and looking up its past success rate on similar episodes Can past performance predict when a model will be right?. Recorded outcomes beat self-perception for people and for models alike.
The broader lesson is that appearance and capability come apart easily. Models trained to imitate ChatGPT fool human evaluators with confident, fluent style while gaining no real capability Can imitating ChatGPT fool evaluators into thinking models improved?. LLMs also prefer their own text roughly in proportion to how well they recognize it Do LLMs favor their own text because they recognize it?. Self-assessment, whether by humans or machines, tends to pick up familiarity and style rather than competence. One caveat: the corpus has only a couple of studies directly on human AI-skill self-reports, so the human side of this comparison rests on a small but consistent evidence base.
Sources 6 notes
A pooled analysis of three studies found a correlation of only .055 between self-reported and objective measures of AI competence, with confidence intervals including zero. This provides no basis for substituting self-assessment for demonstrated performance.
Machine learning identified interpretable traits from coding-agent conversations that explained outcomes beyond prior achievement. However, these traits lacked the stability and transferability required to qualify as learnable human-AI collaboration skills.
LLM self-reports usually reflect human training distributions rather than actual internal processes. However, when a causal chain connects an internal state to accurate reporting—like inferring low temperature from output consistency—genuine lightweight introspection occurs without requiring consciousness.
XConf matches ten-sample self-consistency at a tenth of the cost by retrieving the model's past episodes with similar confidence levels and reading their historical success rates. Ablations show the signal depends entirely on stored outcomes, not on the retrieval prompt itself.
Imitation models fool human evaluators by mimicking ChatGPT's confident, fluent style while failing to improve factuality or generalization on novel tasks. The ceiling is set by base model capability, not fine-tuning method—better fundamentals, not shortcuts, drive real improvement.
Show all 6 sources
Fine-tuning LLMs to recognize their own summaries increased their preference for those summaries in a linear relationship, suggesting recognition capability drives self-preference bias. The authors present this as initial causal evidence, not proof.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- LLM Evaluators Recognize and Favor Their Own Generations
- Beyond Accuracy: The Role of Calibration in Self-Improving Large Language Models
- Confidence Comes from Experience: Experiential Confidence Estimation from Reasoning to Agents
- AI Self-preferencing in Algorithmic Hiring: Empirical Evidence and Insights
- The False Promise of Imitating Proprietary LLMs
- Does It Make Sense to Speak of Introspection in Large Language Models?
- Quantitative Introspection in Language Models: Tracking Internal States Across Conversation
- Reported Confidence in LLMs Tracks Commitment More Than Correctness