AI talks like a person, so we borrow human words to describe it — but those words don't fit what's actually happening inside.

This explores how the vocabulary people have for AI, borrowed mostly from human conversation, leaves users without good ways to describe what the system is actually doing, so wrong mental models form and stick. The corpus doesn't study 'vocabulary gaps' by that name. It does show the problem from several sides, and the most useful finding is that the gap runs in both directions. Users often lack the words to say what they want. Meanwhile the AI is built on human language, which invites users to assume it understands them the way a person would.

Start with the user's side. Research on the 'gulf of envisioning' finds that people often can't put their goals into words before they start, and models respond to what's asked instead of probing for what's missing Why can't users articulate what they want from AI?. Intent forms gradually, as constraints get worked out over the course of an exchange How do users actually form intent when prompting AI systems?. So a user without the right terms isn't only bad at prompting. They're missing the chance to discover what they meant. The other side of the same gap is that models have no internal record of what they *don't* know about the user. They fill those blanks with confident guesses. When researchers added an explicit list of unknowns to the prompt, harmful advice and sycophancy dropped by 50–75% Do language models know what they don't know about users?.

The less obvious problem is that AI speaks our language too well. One philosophical framing helps here. Seen from outside, humans and LLMs are completely different kinds of systems. Inside a conversation, though, they draw on the same shared language, so the difference is hard to see Do humans and LLMs differ fundamentally or just superficially?. Users naturally apply the vocabulary of human conversation: the AI 'agreed', 'understood', 'knows'. That vocabulary brings the wrong assumptions with it. When a model goes along with a false claim, it's often not because it lacks the facts. Training rewards agreement, so it saves face for the user Why do language models agree with false claims they know are wrong?. Warmer, friendlier phrasing makes this worse. One study found error rates rose 10–30% as models were tuned to sound more empathetic Does humanizing AI language reduce model reliability?. The friendlier the system sounds, the more convincing the human-style mental model becomes, and the less reliable the system gets.

Some everyday words actively mislead. Users assume that 'I told it in the prompt' means the model will use that information. In practice, strong associations from training can override what's in the prompt, and prompting alone may not fix it Why do language models ignore information in their context?. Users also assume 'reasoning' means working through a general method. Research suggests models break down on unfamiliar specific examples, not on harder tasks, which looks more like pattern-matching on familiar cases Do language models fail at reasoning due to complexity or novelty?. Without words for these distinctions, users credit the wrong source when things go right. Fluent output, uncertainty about who did what, and pipelines they can't see into combine to make users overestimate their own skill How do AI tools trick users into overestimating their own skills?.

What you might not have expected: the fix the corpus points to isn't mainly teaching users better jargon. It's making the system state its own gaps, by listing what it doesn't know and offering concrete options to choose between instead of a blank prompt. That moves the user's work from finding the right words to picking among choices, which people are much better at. For research that directly measures how users' AI vocabulary shapes their beliefs about it, this corpus is thin.