Full-Duplex Speech Models: 5 Voice AI Myths, Tested

I spent two years building voice assistants on the classic stack: speech-to-text in, language model in the middle, text-to-speech out. Every latency optimization we shipped made it faster and somehow worse. The recent wave of full-duplex speech models — systems like Dessa, a research preview from a small lab at Block that was trained from scratch on nothing but raw human conversation — finally explains why. The problem was never speed. It was architecture. And the amount of folklore floating around about how voice AI "should" work is staggering. So let's do what we did with DeepSeek's inference engine myths: take the common beliefs about conversational speech AI and actually test them against what these new models demonstrate.

Myth 1: Cascaded pipelines just need lower latency

This is the belief I held the longest, and it's the most wrong. The standard architecture — automatic speech recognition (ASR) feeding an LLM feeding text-to-speech (TTS) — has one fatal flaw that no amount of engineering can fix: the first stage deletes the signal the conversation runs on.

When you transcribe speech, you keep the words and discard everything else. Hesitation. Sarcasm. The sigh before "fine." The crack in a voice that tells you someone is about to cry. Your ASR stage converts all of that into a string like "I'm fine" and the LLM downstream has literally no way to know the user is upset. You can bolt on a separate emotion classifier, sure — now you have a fourth model, another 100ms of latency, and a label like sadness: 0.73 that the LLM still treats as a token, not a feeling. It's lossy compression applied to the one part of the channel that carries the most meaning.

There's a second, subtler problem: latency in a pipeline is additive and sequential. ASR can't finish until you stop talking (or it guesses an endpoint), the LLM can't start until ASR finishes, and TTS can't start until the LLM emits tokens. Humans don't work like this. We start formulating a response while the other person is mid-sentence, which is why the average gap between turns in natural conversation is around 200 milliseconds — faster than most cascaded systems can even finish their ASR stage. A full-duplex model that processes audio-in and audio-out continuously, in a single network, sidesteps the whole queue. It's the difference between a relay race and two people just talking.

Verdict: busted. Latency tuning on a cascaded pipeline is optimizing the wrong variable. The architecture itself is the bottleneck.

Myth 2: Text captures everything that matters in conversation

Try this experiment, which I stole from the Dessa team's writeup because it's the cleanest illustration I've seen. Say this sentence out loud seven times, stressing a different word each time: "I didn't say she stole my CUDA code." Seven identical strings of text. Seven completely different meanings — denial of accusation, implication of someone else, quibbling over the verb. Any text-based system sees one input. A speech-native model sees seven.

Prosody — stress, pitch, rhythm, pacing — isn't decoration on top of language. In spoken conversation it is a large part of the language. Laughter mid-sentence signals something different from laughter after it. Backchannels ("yeah", "mm-hm", "got it") carry no lexical content at all, yet a conversation stripped of them feels like talking to a wall. If your system's internal representation is text, you've already lost. The information never enters the model, so no amount of downstream cleverness recovers it. It's the Nyquist theorem applied to UX: sample below the signal's bandwidth and aliasing is permanent.

This has a real safety dimension too, and it's the argument I find most compelling. A voice assistant that hears "I'm in a lot of pain" with flat affect is deaf to urgency in the exact modality where urgency lives. The Dessa team frames this explicitly: as AI systems get more autonomous, being blind to the emotional content of human speech isn't just a UX gap, it's a hazard. Whether or not you buy their model as the fix, the diagnosis is right.

Verdict: busted, and it was never close. Text is a projection of speech, and like all projections it loses a dimension.

Myth 3: Turn-taking is just a voice-activity-detection problem

Ask most engineers how a voice assistant knows when to speak and they'll say "endpointing" — a VAD module waits for N milliseconds of silence, then hands off. This is why every voice assistant you've ever used either interrupts you mid-thought or sits there in agonizing silence after you finish. Silence is a terrible proxy for "your turn." Humans pause mid-sentence to think, trail off invitingly, speed up when they want to keep the floor, and slow down when they're yielding it. Turn-taking is negotiated continuously through prosody, grammar, and context — it's predicted, not detected.

Full-duplex models treat this as a first-class modeling problem. Because they listen and speak simultaneously, they can learn the actual distribution of human conversational timing: when to backchannel, when to jump in to complete someone's stalled sentence (one Dessa demo has it supplying a movie title the human is blanking on), and when to shut up. Interruption isn't an error condition to be suppressed; it's a signal to be modeled. That reframing is the real contribution here, more than any single demo.

But here's where I'll push back on the hype, because the Hacker News commenters have a legitimate point: several of these demos come off as over-eager. When the model laughs before the human finishes a sentence, or steamrolls into a monologue, it reads as a model that learned "interrupting = natural" a bit too well. Predictive turn-taking done perfectly is invisible. Done at 80%, it's more annoying than a polite half-second delay, because a bad interruption feels rude in a way a slow response doesn't. The uncanny valley for conversational timing is real, and nobody has fully crossed it.

Verdict: half-busted. Turn-taking is genuinely a learning problem, not a threshold problem — but the current models prove the approach is right more than they prove it's solved.

Myth 4: You need internet-scale text data to learn language

This one sounds obvious — LLMs consume trillions of tokens, ergo language needs trillions of tokens — until you remember that every human who ever lived learned to speak from a few years of audio exposure with zero text. Babies are the existence proof that the data requirement is much lower than the LLM playbook assumes. Dessa is the machine version of that experiment: trained from scratch, no pretrained weights, no text at all, purely on recorded human conversation. And per the team's claims, it learned language from hundreds of times fewer words than a typical LLM corpus.

Before we crown audio the superior training signal, some honest caveats. First, a spoken word costs thousands of times more bytes than its text form, and speech never repeats — the same sentence said twice is different data. So "fewer words" doesn't mean "fewer bytes"; it means fewer words, which is a claim about how much information speech carries per lexical unit, not about data efficiency in the storage sense. Second, LLM corpora aren't just language — they're code, math, LaTeX, the entire written record of human knowledge. A speech-only model knows a shocking amount for its training diet, but nobody is going to ask it to review your pull request. This connects to a broader pattern we've covered before: benchmarks and demos capture a thin slice of what real work demands, and conversational charm is a different capability axis than reasoning depth.

What the result does establish is genuinely interesting for research: language structure is learnable from acoustics alone, which tells us something real about how much of "language" lives in text conventions versus in speech itself. It also opens practical questions — can speech-native models pick up low-resource languages where audio exists but transcribed text doesn't? That's a thread worth pulling, especially given how much low-resource language work struggles with missing text corpora.

Verdict: mostly busted for spoken language, with an asterisk the size of a data center. Speech-alone learning works; it just optimizes for a different thing than text-alone learning.

Myth 5: A realistic voice means a smart — or safe — system

This is the myth that should worry you, and the comment section saw it immediately: "impersonation — that's what this is for." Harsh, but directionally correct. Fluency and intelligence are orthogonal, and speech fluency is the more persuasive of the two. A model that gasps "oh my god" at your anniversary story with perfect teary timing is doing something technically remarkable, but nothing about that implies it reasons well, knows facts reliably, or won't confidently make things up in a warm, empathetic voice. If anything, expressive delivery masks shallow reasoning — we're wired to trust confident, emotionally attuned speakers. That's the whole basis of con artistry.

The threat model writes itself. Voice cloning plus full-duplex conversational ability plus a spoofed caller ID is a fraud kit that defeats the instinct most people rely on — "I'd know if it wasn't them." One commenter's mitigation is worth adopting: a challenge question about shared memory for any suspicious call from someone you know. It feels paranoid. It's going to become standard practice. To the team's credit, they're explicitly inviting safety researchers to poke at the model before a wider release, which is more than most labs do. But the technology itself is dual-use in the most literal sense, and no amount of framing changes that.

There's also a cost story buried in here that cuts the other direction, in a good way. The claim is that inference runs faster than real time on a MacBook Pro with an M4 Max, and that training fit on a single NVIDIA GB200 NVL72 system. If those numbers hold up, that's remarkable: frontier-feeling conversational AI that isn't gated behind a hyperscale cluster. Local inference matters for privacy (your voice never leaves the device), for latency (no network hop in the middle of a duplex stream), and for who gets to build on this stuff. The efficiency trend in small capable models is one we've flagged before with flash-tier LLMs — the interesting action in inference is increasingly at the "runs on hardware you own" end of the spectrum.

Verdict: busted, and it's the most dangerous myth on the list. Treat vocal realism as a UI property, not a capability or trust signal — and update your personal security habits accordingly.

What this actually changes

Strip away the "most authentic ever built" marketing (the title got moderated on Hacker News for a reason) and what's left is still significant: an end-to-end speech model is a categorically different thing from a pipeline, and the differences compound. Emotion stays in the loop. Timing becomes learned behavior instead of a hand-tuned threshold. Interruptions become data. None of that is achievable by making ASR 50ms faster.

My pragmatic take, as someone who shipped the old architecture: cascaded pipelines aren't dead. For transactional voice interfaces — check a balance, book an appointment, control a device — they're fine, they're debuggable, and each stage is independently swappable. Text intermediates give you logging, redaction, and policy enforcement for free, which matters enormously in production. But for anything where the conversation is the product — tutoring, companionship, accessibility tools, negotiation practice — the pipeline is a dead end, and end-to-end audio is the only architecture that can ever feel natural. Choose based on whether your users are talking to the machine or through it.

The open questions are the fun ones. How well does in-context learning work when the context is audio? Can these models hold long conversational state without the crutch of a text transcript? Does emotional fluency make failures more harmful because users trust the delivery? And the uncomfortable one: once this quality of impersonation runs locally on a laptop, what does verification of human identity even look like? The labs that answer those honestly will matter more than the ones with the slickest demo reel.

Voice is the oldest interface humans have. We've spent forty years forcing it through a text-shaped hole and wondering why it felt wrong. Turns out the hole was the problem.