Do patient interruptions break clinical voice agent safety?
When patients interrupt mid-turn, do conversational AI agents preserve critical safety content? Testing four LLMs on simulated cataract follow-up calls reveals whether interruption robustness can be reliably measured.
A transcript-based evaluation of post-operative cataract follow-up calls tests four deployment-oriented, non-reasoning LLMs (Llama 3.1 8B, GPT-5.4 mini, Claude Haiku 4.5, and Gemini 2.5 Flash) across history-taking (information gathering) and FAQ (information provision) cells, scoring whether an agent preserves clinically required content after a simulated patient interruption. "In the gathering cells, target-question failure varied across models; in the provision cells, where arms are directly comparable, failure rose for every model." The sharpest result: "competitive FAQ interruption produced 30/30 provision-coverage failures for all four models (Wilson 95% CI: 88.6–100.0%; baseline 0/30 for three, 4/30 for Llama)." A brief apology marker ("sorry to interrupt") "shifts recovery by tens of percentage points, inconsistently across models, and for one it reduces recovery." The authors conclude "interruption robustness therefore cannot be a single score: evaluation must be content-grounded, reported per cell, and matched to the deployment's interruption profile."
The mechanism follows from the cascaded architecture (speech-to-text → LLM → text-to-speech): once an interruption truncates the agent's turn, "the downstream recovery problem is textual: the LLM receives a partial dialogue history and must decide what clinically relevant content to preserve." The paper adapts conversation-analytic overlap categories (Sacks et al., 1974; Schegloff, 2000) into three operational types — transitional sub-unit (a later clause in a multi-unit turn is dropped), recognitional (the patient answers a truncated question stem under a different interpretation), and competitive (the patient redirects mid-turn, "far from any natural completion point"). Competitive overlap is the costliest because, per Schegloff, it is structurally the most "interruptive" placement: "the patient's next interactional move displaces the unfinished clinical content, and no model in our panel recovers it under the standard agent prompt." Defensive prompting barely helps: the strongest variant "leaves three of four models at 100% failure and only moves the fourth to 90%." A review of 35 real deployed-agent cataract calls found 20 contained at least one overlap during a safety-critical sequence, mostly transitional sub-unit (17 of 27 instances), establishing that the phenomenon is not a simulator artifact.
Against the library, the result exposes a blind spot in evaluation designs that assume cooperative turn-taking. Can conversational AI safely take patient histories without supervision? reports zero safety interventions needed across 100 real patient conversations, but that count is only legible because supervisors watched exchanges that proceeded turn by turn; this paper's point is that a zero stops metric cannot surface interruption-induced loss, since the failure here is not a flagged safety violation but a silently dropped clinical obligation. The mechanism also echoes Why do language models fail in gradually revealed conversations?: both findings turn on a model committing to an interpretation of a partial turn and failing to recover full content once context arrives incompletely or out of order, though this paper locates the cause in interactional structure (where the cut lands) rather than instruction-revelation order. It also qualifies Why do language models lose performance in longer conversations?: if the failure were purely an intent-alignment gap, a defensive "resume after interruption" prompt should close more of the gap than the reported 0–10 percentage points it closes here. And it parallels Can language models truly understand therapeutic ruptures? in showing models handling an explicit, surface-visible cue (the apology marker) inconsistently rather than reliably, with effects running in opposite directions for different models.
The evaluation is explicitly a first, text-tractable study: it does not model prosody, acoustic cues, or wall-clock timing that shape real interruption recognition, so findings generalize only to "the cascaded-truncation class of voice architectures," not to full-duplex systems that preserve complete agent turns. It covers a single pathway (cataract follow-up) in English only, and the authors say reference content points "would need to be re-derived per pathway." The paper's own recommendation is that robust recovery "likely requires interruption-aware state tracking of unresolved questions and advice points, not only stronger prompts" — the apology-marker result suggests that surface conversational politeness cues are not a reliable substitute for that state tracking.
Inquiring lines that read this note 9
This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.
How can conversational AI develop appropriate initiative and goal awareness?- Can voice agents learn when to speak unprompted without seeming disruptive?
- How common are patient interruptions in real deployed voice agents?
- Do conversation analysis categories predict AI interruption robustness?
- Can individual customization of backchannel frequency improve voice agent naturalness for users?
- How does wait time and interruption frequency interact with backchannel effectiveness?
Related concepts in this collection 4
This note in its neighbourhood — explore the map, then jump to a related concept in the list below.
Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph
-
Can conversational AI safely take patient histories without supervision?
A feasibility study tested whether an LLM-based system could conduct real clinical interviews with urgent-care patients without requiring safety interventions. Understanding AI safety in unsupervised clinical settings matters for potential deployment.
contrast: a "zero stops" safety count under cooperative turn-taking cannot surface this paper's interruption-induced content loss
-
Why do language models fail in gradually revealed conversations?
Explores why LLMs perform 39% worse when instructions arrive incrementally rather than upfront, and whether they can recover from early mistakes in multi-turn dialogue.
parallel mechanism: committing to a partial-turn interpretation and failing to recover full content once context is incomplete
-
Why do language models lose performance in longer conversations?
Does multi-turn degradation stem from fundamental model limitations, or from misalignment between what users mean and what models assume? Understanding the root cause could guide better solutions.
qualifies: defensive resume-after-interruption prompting here closes only 0-10pp, less than a pure intent-alignment account would predict
-
Can language models truly understand therapeutic ruptures?
When LLMs match expert labels for therapeutic ruptures, are they demonstrating genuine clinical understanding or relying on surface-level linguistic patterns? This matters because high identification scores may mask fundamentally different reasoning.
parallel: models handle an explicit surface cue (the apology marker) inconsistently rather than reliably
Related papers in this collection 8
Papers most semantically related to this note, ranked by cosine similarity in the embedding space.
- When Patients Cut In: Extending Clinical Conversational AI Safety to Interruptions
- Toward Enabling Natural Conversation with Older Adults via the Design of LLM-Powered Voice Agents that Support Interruptions and Backchannels
- A prospective clinical feasibility study of a conversational diagnostic AI in an ambulatory primary care clinic
- Towards Conversational Diagnostic AI
- Clinical knowledge in LLMs does not translate to human interactions
- Tailored Conversations beyond LLMs: A RL-Based Dialogue Manager
- Can LLMs identify and repair ruptures? Comparison between clinician practices and LLM behaviors
- AI-based Clinical Decision Support for Primary Care: A Real-World Study
Original note title
clinical voice agents lost safety-critical content under patient interruption — competitive overlap caused near-total failure across four LLMs