SYNTHESIS NOTE
Topics›Speech Voice›this note

Do patient interruptions break clinical voice agent safety?

When patients interrupt mid-turn, do conversational AI agents preserve critical safety content? Testing four LLMs on simulated cataract follow-up calls reveals whether interruption robustness can be reliably measured.

Synthesis note · 2026-10-22 · sourced from Speech Voice

A transcript-based evaluation of post-operative cataract follow-up calls tests four deployment-oriented, non-reasoning LLMs (Llama 3.1 8B, GPT-5.4 mini, Claude Haiku 4.5, and Gemini 2.5 Flash) across history-taking (information gathering) and FAQ (information provision) cells, scoring whether an agent preserves clinically required content after a simulated patient interruption. "In the gathering cells, target-question failure varied across models; in the provision cells, where arms are directly comparable, failure rose for every model." The sharpest result: "competitive FAQ interruption produced 30/30 provision-coverage failures for all four models (Wilson 95% CI: 88.6–100.0%; baseline 0/30 for three, 4/30 for Llama)." A brief apology marker ("sorry to interrupt") "shifts recovery by tens of percentage points, inconsistently across models, and for one it reduces recovery." The authors conclude "interruption robustness therefore cannot be a single score: evaluation must be content-grounded, reported per cell, and matched to the deployment's interruption profile."

The mechanism follows from the cascaded architecture (speech-to-text → LLM → text-to-speech): once an interruption truncates the agent's turn, "the downstream recovery problem is textual: the LLM receives a partial dialogue history and must decide what clinically relevant content to preserve." The paper adapts conversation-analytic overlap categories (Sacks et al., 1974; Schegloff, 2000) into three operational types — transitional sub-unit (a later clause in a multi-unit turn is dropped), recognitional (the patient answers a truncated question stem under a different interpretation), and competitive (the patient redirects mid-turn, "far from any natural completion point"). Competitive overlap is the costliest because, per Schegloff, it is structurally the most "interruptive" placement: "the patient's next interactional move displaces the unfinished clinical content, and no model in our panel recovers it under the standard agent prompt." Defensive prompting barely helps: the strongest variant "leaves three of four models at 100% failure and only moves the fourth to 90%." A review of 35 real deployed-agent cataract calls found 20 contained at least one overlap during a safety-critical sequence, mostly transitional sub-unit (17 of 27 instances), establishing that the phenomenon is not a simulator artifact.

Against the library, the result exposes a blind spot in evaluation designs that assume cooperative turn-taking. Can conversational AI safely take patient histories without supervision? reports zero safety interventions needed across 100 real patient conversations, but that count is only legible because supervisors watched exchanges that proceeded turn by turn; this paper's point is that a zero stops metric cannot surface interruption-induced loss, since the failure here is not a flagged safety violation but a silently dropped clinical obligation. The mechanism also echoes Why do language models fail in gradually revealed conversations?: both findings turn on a model committing to an interpretation of a partial turn and failing to recover full content once context arrives incompletely or out of order, though this paper locates the cause in interactional structure (where the cut lands) rather than instruction-revelation order. It also qualifies Why do language models lose performance in longer conversations?: if the failure were purely an intent-alignment gap, a defensive "resume after interruption" prompt should close more of the gap than the reported 0–10 percentage points it closes here. And it parallels Can language models truly understand therapeutic ruptures? in showing models handling an explicit, surface-visible cue (the apology marker) inconsistently rather than reliably, with effects running in opposite directions for different models.

The evaluation is explicitly a first, text-tractable study: it does not model prosody, acoustic cues, or wall-clock timing that shape real interruption recognition, so findings generalize only to "the cascaded-truncation class of voice architectures," not to full-duplex systems that preserve complete agent turns. It covers a single pathway (cataract follow-up) in English only, and the authors say reference content points "would need to be re-derived per pathway." The paper's own recommendation is that robust recovery "likely requires interruption-aware state tracking of unresolved questions and advice points, not only stronger prompts" — the apology-marker result suggests that surface conversational politeness cues are not a reliable substitute for that state tracking.

Inquiring lines that read this note 9

This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.

How can conversational AI develop appropriate initiative and goal awareness? Is mimicking human communication the best design approach for conversational AI? What makes decomposed prompts more reliable than monolithic instructions? Do safety evaluations reliably measure real deployed AI safety risks? Why do LLM outputs diverge from their knowledge and reasoning?

Related concepts in this collection 4

This note in its neighbourhood — explore the map, then jump to a related concept in the list below.

Concept map
12 direct connections · 97 in 2-hop network ·medium cluster Open in graph ↗

Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph

your link semantically near linked from elsewhere

Related papers in this collection 8

Papers most semantically related to this note, ranked by cosine similarity in the embedding space.

Original note title

clinical voice agents lost safety-critical content under patient interruption — competitive overlap caused near-total failure across four LLMs