What expectations does human conversation activate that AI should avoid triggering?
This explores which deep reflexes of human dialogue — proactivity, word-matching, evolving shared context, the sense of a real partner on the other side — AI quietly fires off in users without being able to deliver on them, and which it might therefore be wiser not to imitate.
This explores which expectations human conversation activates that AI should avoid triggering — the ingrained reflexes of dialogue that AI inherits the surface of but not the substance. The corpus frames the core problem sharply: conversational interfaces switch on a lifetime of communication competencies that were built for talking with other minds, but AI doesn't actually communicate, so the very fluency of the design produces failures that feel like user error while really originating in design choices Why do users fail with AI interfaces designed like conversations?. At a deeper level, the thing arriving on your screen isn't an utterance at all — it's 'event-residue' carrying the communicative markers of its training data, which the human then animates into a pseudo-exchange that has real structure only on the human side Does AI generate genuine utterances or just text patterns?. The expectation to avoid triggering, then, is the expectation that there is a someone there reciprocating.
One concrete expectation is that context is something the two of you build together and can quietly renegotiate as you go. Human dialogue evolves cooperatively; a prompt, by contrast, bundles utterance, context, and role assignment into one static frame the model can't renegotiate — so when you want to pivot mid-conversation, the implicit adjustment a human partner would make becomes an explicit re-prompt you have to perform yourself How do prompts reshape the role of context in AI conversation?. Relatedly, humans expect their partner to drift toward their words — lexical entrainment, the gradual mirroring of vocabulary that builds rapport and clarity. Current systems largely don't do this, so they keep activating an expectation of convergence they don't meet Why don't conversational AI systems mirror their users' word choices?.
There's a subtler trap around social presence. You don't need a rich pile of cues to make a system feel like a social actor — a single primary cue like a voice or a face is enough to switch on the full social response, while stacking secondary cues adds little Do more social cues always make AI feel more present?. That means presence is cheap to evoke and easy to over-evoke: a small design choice can summon the whole apparatus of human social expectation, and then the gaps show. The starkest version of the gap is normative — AI can predict social appropriateness more accurately than any individual human, yet it structurally cannot participate in the community processes that create and validate norms in the first place Can AI predict social norms better than humans?. Knowing the norms from the outside is not the same as being a member who helps make them, and conversational framing invites users to assume the latter Can AI learn social norms better than humans?.
What's interesting is that the corpus doesn't conclude 'make AI less conversational.' It splits the conversational reflexes into ones worth honoring and ones worth dropping. The reflexes worth honoring are the genuinely cooperative ones: proactively offering relevant information without being asked, which mirrors Grice's conversational maxims and can cut dialogue turns by up to 60% — yet is nearly absent from AI training data and benchmarks Could proactive dialogue make conversations dramatically more efficient?. The reason it's missing is structural: standard RLHF optimizes for immediate, single-turn helpfulness, which actively trains models to answer passively rather than ask the clarifying question a good human interlocutor would Why do language models respond passively instead of asking clarifying questions?. Conversation analysis even offers a formal vocabulary — 'insert expansions' — for when an agent should pause and probe intent instead of silently chaining tools toward a misread goal When should AI agents ask users instead of just searching?.
So the synthesis cuts against the obvious answer. The expectations AI should avoid triggering aren't the cooperative behaviors of conversation — those are exactly what it's failing to deliver. What it should avoid is the false promise underneath the cooperative surface: the expectation of a reciprocating partner who shares context, co-creates norms, and means what it says. The unexpected lesson is that good conversational AI may need to be *more* like real conversation in its mechanics (asking, entraining, anticipating, knowing when to interrupt — and timing that interruption, since support helps or harms depending on timing and scale, not just type When and how much should AI interrupt human reasoning?) while being honest enough not to cash the social check that fluency writes.
Sources 11 notes
AI interfaces that use conversational design conventions trigger users' lifelong communication skills, but AI doesn't actually communicate. This mismatch causes interaction failures that feel like user error but originate in design.
AI output carries communicative markers inherited from training data but lacks the event structure that produces actual utterances. Users supply the missing orientation through interpretive labor, creating a pseudo-event with structure only on the human side.
LLM prompts bundle utterance, context assignment, and role specification into a single static frame the model cannot renegotiate, unlike human dialogue where context evolves cooperatively. This makes mid-conversation pivots require explicit re-prompting rather than implicit adjustment.
Response generation models fail to adapt vocabulary toward users' lexical choices, a phenomenon central to human rapport and clarity. Post-training via DPO on coreference-identified preferences can teach models in-context convention formation.
Research shows individual primary cues like voice or appearance are sufficient to evoke social-actor presence, while multiple secondary cues cannot. Quality of cues matters more than quantity in driving social responses.
GPT-4.5 outperforms all individual humans at predicting social appropriateness, yet structurally cannot enter the community processes that establish and validate norms. This reveals a critical gap between pattern-matching and authentic participation in knowledge-making.
GPT-4.5 outperformed every individual human at judging social appropriateness across 555 scenarios, challenging the theory that embodied cultural experience is necessary. However, all AI models share identical systematic errors on unwritten norms.
Simulations show proactivity—providing relevant information without being asked—cuts dialogue turns by 60% in medium-complexity domains. This behavior mirrors human conversation and Grice's maxims but is almost entirely absent from AI datasets and research benchmarks.
CollabLLM demonstrates that standard RLHF training optimizes for immediate helpfulness, discouraging models from asking clarifying questions or offering multi-turn insights. Multi-turn-aware rewards that estimate long-term interaction value enable active intent discovery and genuine collaboration.
Tool-enabled LLMs drift from user intent through silent tool chaining. Conversation analysis reveals insert-expansions—clarifying intent, scoping responses, enhancing appeal—as a formal framework for proactive user consultation that prevents misunderstanding instead of recovering from it.
Research identifies three orthogonal axes—type, timing, and scale—that jointly determine whether cognitive support helps or harms. Most explainable AI optimizes type alone, leaving timing and scale as implicit defaults, missing where real impact occurs.