Can one streaming model handle turn-taking better than cascaded ASR-LLM-TTS?

This explores whether one unified streaming model beats the classic cascade — speech-to-text, then LLM, then text-to-speech — at the hard part of real conversation: knowing *when* to speak. The corpus leans clearly toward the unified model, and the reason is more interesting than raw speed. The cascade's deepest weakness is that it treats turn-taking as plumbing between modules, when in fact timing *is* the conversation. Can a single model learn when to speak and respond? makes the case directly: Wan-Streamer folds language, audio, and video into one interleaved causal token stream, so response timing and turn management are *learned* jointly inside a single Transformer rather than engineered as a separate barge-in detector or end-of-utterance heuristic. Turn-taking emerges as behavior, and latency drops below a second.

The second argument against the cascade is error accumulation, and it's quantifiable. Why do dialogue systems need probabilistic reasoning? shows that real-world speech recognition runs 15–30% word error in noisy rooms — and in a cascade that error is handed downstream as if it were clean text. The older fix was to never commit: POMDP dialogue systems carried a *belief distribution* over what the user meant instead of one transcript. A streaming model inherits that spirit for free, since it never has to flatten audio into a single discrete string before reasoning. Interestingly, this rhymes with how transformers handle knowledge generally — Do transformer models store knowledge or generate it continuously? frames model cognition as continuous flowing activation rather than retrieval from fixed storage, which is exactly the property you want when sound, meaning, and timing should stay entangled rather than be serialized through a text bottleneck.

But here's the thing the question doesn't ask, and where the corpus gets sharp: better turn-*timing* does not automatically buy you better turn-*taking* in the conversational sense. A model can know precisely when to speak and still be a bad interlocutor. Why do language models fail in gradually revealed conversations? and Why do AI assistants get worse at longer conversations? document a brutal failure: accuracy falls from ~90% on a single-shot instruction to ~65% across a natural multi-turn exchange, because models lock onto early guesses and can't course-correct. That's an architecture-agnostic flaw — going full-duplex won't fix it, and might even worsen it by encouraging the model to commit and speak *faster*.

The corpus traces this to training objectives, not pipeline shape. Why do language models respond passively instead of asking clarifying questions? and Why do language models lose performance in longer conversations? argue that standard RLHF rewards immediate helpfulness, which teaches models to answer prematurely instead of asking a clarifying question — the single most natural use of a turn in real talk. And good turn-taking involves social mechanics the cascade-vs-unified debate ignores entirely: Why don't conversational AI systems mirror their users' word choices? notes models don't drift toward a user's vocabulary the way human partners do, and Can LLMs truly update shared conversational common ground? shows they treat the opening prompt as a fixed frame and can't symmetrically update shared assumptions mid-conversation.

So the honest answer: yes, one streaming model handles the *mechanics* of turn-taking better — lower latency, emergent timing, no error-amplifying text bottleneck. But "turn-taking" as a human would judge it is part timing and part intent-tracking, entrainment, and shared-ground maintenance — and those live in training objectives and conversational competence, not in whether you used one box or three. The streaming model removes the cascade's structural penalties; it doesn't, by itself, make the model a good conversational partner.