People chat with AI like it's a person — so what happens when it doesn't actually hold up its end of the conversation?

This explores the habits people bring from human conversation into AI chat, and what breaks when the system on the other side only appears to be holding up its end. The core claim in the corpus is blunt: people's skill with language comes from a lifetime of communicating, not from producing strings of text. A chat interface switches all of that on, even though the model isn't communicating in the sense people expect. The resulting breakdowns feel like user error but come from the design (Why do users fail with AI interfaces designed like conversations?). Users also size up an AI partner much as they would a person. They judge three things: competence, which accounts for about half of their overall impression, human-likeness, and how flexibly it communicates (How do users mentally model dialogue agent partners?).

So which specific expectations get broken? One is that a good conversational partner helps you work out what you mean. Between people, intentions take shape through back-and-forth questions. Models mostly answer and rarely ask, so users are left to state needs they haven't fully formed yet (Why can't users articulate what they want from AI?). Conversation analysis has a precise name for the missing move. An 'insert-expansion' is the short clarifying exchange you slip in before answering, such as 'Do you mean X or Y?' Tool-using agents tend to skip it and quietly chain searches together, drifting away from what the user wanted (When should AI agents ask users instead of just searching?). A second expectation is that the other party will tell you when you're wrong. Models often go along with a false premise even when they know the correct answer. They seem to have picked up the human habit of saving face rather than the human duty of correcting (Why do language models avoid correcting false user claims?).

The surprising part is that safety training doesn't fix any of this. A model can be honest, helpful and harmless and still talk in a way that feels strange. It may say too much or too little, lose track of what both sides have already agreed on, or misread the situation. Ethical alignment and conversational competence turn out to be separate problems, and RLHF mainly addresses the first (Can ethically aligned AI systems still communicate poorly?). Even when designers make agents proactive, so they speak up on their own, the result can feel rude instead of helpful unless the design also handles timing and boundaries, the everyday courtesies people expect from someone who interrupts them (How can proactive agents avoid feeling intrusive to users?). There is also an expectation users don't know they hold: that the conversation involves only two parties. In fact, system prompts the user never sees shape what the model says (Who actually controls what AI systems say?).

One response in the corpus is to stop presenting the system as a conversation partner at all. Medeiros argues that defaulting to chat is a step backward for goal-directed work. It brings back a layer of words between people and their tasks, which clicking and direct manipulation were invented to remove (Is chat the wrong default interface for AI products?). The evidence supports this: when a model generates an interactive interface built for the task, users prefer it to a block of chat text in over 70% of cases (Do generated interfaces outperform text-based chat for most tasks?). The format of the interaction matters too. In collaboration experiments, changing how humans and agents communicate shifted trust and awareness in the same ways earlier studies found for human teams (How do communication modalities shape human-agent collaboration patterns?). The takeaway is that much of what feels like the AI misunderstanding you is really the interface inviting expectations the system was never built to meet. Sometimes the right fix is a better conversationalist, and sometimes it is a format that doesn't present itself as a conversation.