Voice assistants chain three AI steps — listening, thinking, speaking — and each one's delay adds to the next, making replies feel sluggish.

This explores why voice assistants built as a three-stage chain (speech-to-text, then an LLM, then text-to-speech) feel slow, and how each stage's delay stacks on the next. The retrieved material doesn't cover this pipeline directly. None of these notes measures ASR or TTS latency, so treat what follows as the nearby ideas the collection offers, not a direct answer.

The basic mechanism is simple. In a strict cascade, each stage waits for the one before it to finish. The system has to decide you've stopped talking, transcribe what you said, generate the reply, and only then start speaking it. The middle stage is often the slowest because LLMs generate text one token at a time. That's why work on faster decoding matters here. Can diffusion language models match autoregressive inference speed? describes a hybrid that generates text block by block, with tokens decoded in parallel inside each block, while reusing cached computation. It's one example of shrinking the bottleneck in the middle of the chain. A related constraint shows up in Can LLMs handle multiple tasks at once during inference?: once the first token is produced, the model commits to a single path. That first token largely sets how quickly the system can begin to answer.

The less obvious point is that the usual fix for latency, streaming, creates its own risk. To cut delay, designers overlap the stages: the LLM starts working on a partial transcript, and the speech engine starts talking before the reply is finished. But the multi-turn research shows that LLMs given information piece by piece tend to lock in early guesses they can't undo. Performance drops about 39% when instructions arrive gradually (Why do language models fail in gradually revealed conversations?, Why do AI assistants get worse at longer conversations?). A voice system that answers before you finish your sentence is running that same experiment in real time. Trimming latency can mean committing to a wrong reading of what you asked.

The chain also adds a cost beyond time. Because the LLM only sees the transcript, anything the speech recognizer drops (hesitation, emphasis, a corrected word) is gone before reasoning starts. And because LLMs tend to treat the opening framing as fixed (Can LLMs truly update shared conversational common ground?), a misheard first sentence can shape everything after it. That's why some designers prefer end-to-end speech models over cascades. To go deeper on cascade latency itself, look in the collection's speech and voice conversation area rather than these results.