Do people glance at their partner more when a conversation starts to go off the rails? The evidence points that way.
Does partner-directed gaze increase at moments of communicative difficulty?
This explores whether people look at their conversation partner more when understanding is breaking down, treating gaze as a visible sign of trouble.
This explores whether people look at their conversation partner more when understanding is breaking down. The corpus doesn't test that moment by moment, but it has the mirror image. In two collaborative-task datasets (MapTask and Mundex), stretches where the pair had reached shared understanding coincided with more gaze at the task, less gaze at the partner, and lower gaze entropy, meaning looking that was less scattered (Does gaze reveal whether people have achieved common ground?). Read in reverse, partner-directed gaze is higher when common ground is shaky, which is the direction your question points. That is an inference from the pattern, not a measurement of how people react to a specific misunderstanding.
The finding also changes what looking at your partner means. When things are going well, eyes go to the map or the task, and the partner is the one you check on when something feels off. The effect was consistent but modest, and strongest for task leaders. So it's a tilt in where people look, not a reliable alarm bell, and a single glance can't tell you whether someone is confused.
The corpus has a related case of a subtle non-verbal signal reading opposite to intuition. In therapy transcripts, patient filler pauses signal relaxed communication and a stronger alliance, not hesitation (Does therapist self-reference language predict weaker therapeutic alliance?). Difficulty and comfort often show up in small cues that don't look the way we'd guess.
The AI angle is that this channel is missing from most systems. The gaze paper notes the signal is invisible to text-only systems, so a chat model can't see the look that says the other person hasn't followed. Preference optimization also works against the verbal repair that would replace it. RLHF-trained models produce grounding acts, such as clarifying questions and understanding checks, at 77.5% below human levels (Does preference optimization harm conversational understanding?). Frameworks like collaborative rational speech acts model the move from partial to shared understanding by tracking both speakers' beliefs, but only from what is said (Can dialogue systems track both speakers' beliefs across turns?). The corpus has no note on gaze around specific repair moments, such as a clarification request or a corrected misunderstanding. That is the missing piece for a firm yes.
Sources 4 notes
Across two corpora, aligned understanding coincided with more task-directed gaze, less partner-directed gaze, and lower gaze entropy—especially for task leaders. The effect was consistent but modest, and invisible to text-only systems.
High frequency of therapist 'I' usage correlates with lower patient-reported alliance and reduced trusting behavior in validated behavioral tasks. Patient non-fluency markers like filler pauses, conversely, signal relaxed communication and stronger alliance.
RLHF optimizes models for single-turn helpfulness by rewarding confident responses over clarifying questions and understanding checks. This preference alignment systematically reduces grounding acts by 77.5% below human levels, creating an alignment tax where models appear helpful but fail silently in multi-turn contexts.
CRSA integrates rate-distortion theory with RSA to enable bidirectional belief tracking across dialogue turns. Demonstrated on referential games and doctor-patient dialogues, it captures progression from partial to shared understanding, providing the information-theoretic framework that token-level LLM systems lack.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Intent Mismatch Causes LLMs to Get Lost in Multi-Turn Conversation
- Grounding Gaps in Language Model Generations
- Collaborative Rational Speech Act: Pragmatic Reasoning for Multi-Turn Dialog
- A natural language processing approach reveals first-person pronoun usage and non-fluency as markers of therapeutic alliance in psychotherapy
- Gaze as Evidence for Common Grounding: A Cross-Corpus Analysis of MapTask and MUNDEX
- Consistently Simulating Human Personas with Multi-Turn Reinforcement Learning
- DR-HAI: Argumentation-based Dialectical Reconciliation in Human-AI Interactions
- Rewards-in-Context: Multi-objective Alignment of Foundation Models with Dynamic Preference Adjustment