{"id":23248,"date":"2026-05-06T16:55:43","date_gmt":"2026-05-06T12:55:43","guid":{"rendered":"https:\/\/krisp.ai\/blog\/?p=23248"},"modified":"2026-05-06T16:58:34","modified_gmt":"2026-05-06T12:58:34","slug":"viva-2-0-ai-infrastructure-for-voice-ai-agents","status":"publish","type":"post","link":"https:\/\/krisp.ai\/blog\/viva-2-0-ai-infrastructure-for-voice-ai-agents\/","title":{"rendered":"Introducing Krisp VIVA 2.0: Voice Infrastructure for Voice AI Agents"},"content":{"rendered":"

Every voice AI demo works. Production doesn’t.<\/p>\n

You’ve seen it happen. A voice agent sounds great in the lab. Crisp audio, perfect timing, natural flow. Then it ships. Someone calls from a busy airport. Their kid is screaming in the background. A bad cell connection mangles the audio. The agent talks over the caller, ignores a real interruption, or gets confused by a siren outside the window.<\/p>\n

 <\/p>\n

This is happening everywhere. Voice agent usage grew 9x in 2025. Over 150 companies are building them. Twenty-two percent of Y Combinator’s latest cohort is voice-first. The market crossed $22 billion and is growing at 35% a year. Everyone is building, and everyone is hitting the same wall.<\/p>\n

Two problems keep voice agents from working in production. Neither is new. Both are unsolved, until now.<\/p>\n

\n

The audio problem<\/h2>\n

Real-world voice sounds nothing like a demo room. There’s background noise, other people talking, cheap mics, room echo, feedback loops, and codec compression that chews up the signal before it even reaches your model.<\/p>\n

 <\/p>\n

This breaks things in predictable ways. Noise pushes word error rate from around 5% to 15\u201330% or worse. Background voices trick the bot into thinking someone is speaking when they’re not. On phone calls, the agent’s own voice bounces back into the mic and triggers self-interruption loops.<\/p>\n

It’s not an edge case. It’s every call.<\/p>\n

The conversation problem<\/h2>\n

Even with perfectly clean audio, voice agents still feel off. Human conversation runs on a thousand tiny cues that we pick up without thinking.<\/p>\n

We know when someone is about to finish a thought. We know that “mhm” means “keep going” and not “stop, I have something to say.” We can tell the difference between a pause that means someone is thinking and a pause that means they’re done. Nobody teaches us this. We just feel it.<\/p>\n

 <\/p>\n

Voice agents don’t feel any of it. Most run on one simple rule: when you go quiet, they start talking. Everyone on the other end, whether they’re booking a flight, checking a prescription, or disputing a charge, can tell something is off immediately.<\/p>\n

 <\/p>\n

From reactive to predictive<\/h2>\n

Every voice agent out there today is reactive. It waits for silence, then talks. It hears a sound, then stops. It takes whatever audio it gets, clean or not, and hopes for the best.<\/p>\n

 <\/p>\n

Human conversation doesn’t work that way. It’s predictive. We don’t wait for total silence to know it’s our turn. We don’t stop talking every time someone makes a sound. We’re always reading the signal, anticipating what’s coming next.<\/p>\n

 <\/p>\n

Krisp has spent eight years on this. Not in a lab, but in production. We’ve processed over a trillion minutes of voice traffic across real environments, real devices, real noise. Two-time Webby Award winner for technical achievement. We started with human-to-human communication, powering noise cancellation<\/a> for millions of users. Then we launched VIVA<\/a> for human-to-AI, bringing voice isolation<\/a>, voice activity detection, and turn taking<\/a> to production voice agents at scale.<\/p>\n

 <\/p>\n

VIVA 2.0<\/strong> takes the next step. It doesn’t just clean the audio and hand it off. It understands the conversation. One SDK. Server-side. Sits in the audio pipeline before speech-to-text. Everything downstream gets better.<\/p>\n

 <\/p>\n

This isn’t theory. VIVA is already running inside Daily, Vapi, LiveKit, Vodex, Ultravox, and the world’s largest AI labs. Teams using VIVA have seen 3.5x better turn-taking accuracy, 50% fewer dropped calls, and 30% higher customer satisfaction scores. We process over 10 billion minutes of voice AI traffic a year and growing.<\/p>\n

 <\/p>\n

\n
\n Get Access<\/a>\n <\/div>\n<\/div>\n

What’s in VIVA 2.0<\/h2>\n

Voice Isolation v3: isolate the speaker, improve WER<\/h3>\n

That 15\u201330% word error rate isn’t a small problem. It means your agent hears “I need to cancel my Thursday flight” as “I need to cancel my first day flight” and acts on it. Every misheard word makes things worse downstream. The LLM reasons on bad input, the response goes sideways, the user has to repeat themselves, and trust in the agent drops fast.<\/p>\n

 <\/p>\n

Voice Isolation v3 is a ground-up rebuild of our core engine. It isolates the primary speaker’s voice from everything else \u2014 background noise<\/a>, other voices, room echo, and codec artifacts \u2014 and delivers cleaner audio to your STT pipeline, directly improving word error rate. Works across languages and accents. This is the foundation everything else in VIVA builds on.<\/p>\n

Turn Prediction v3: knowing when to speak<\/h3>\n

Without end-of-turn prediction, bots just wait for silence. The user stops talking, the bot counts a few seconds of quiet, then responds. This is why talking to most voice agents feels slow and robotic.<\/p>\n

 <\/p>\n

Turn Prediction v3 works completely differently. Instead of counting silence, it listens to the music of the speech, the intonation, rhythm, how the sentence is shaped, and predicts the end of turn in a fraction of a second. V3 catches 47% more true turn-shifts within the first 200 milliseconds compared to v2, without more false positives. The bot just responds at the right moment, and the conversation feels natural.<\/p>\n

 <\/p>\n

Now multilingual: English, German, French, Spanish, Hindi, Finnish, Italian, Portuguese, Chinese, Japanese, Korean, Russian, and more. Runs on CPU, ships at 30 MB, works purely on audio with no transcription needed.<\/p>\n

 <\/p>\n

We tested Turn Prediction v3 against every major solution available today:<\/p>\n\n\n\n\n\n\n\n\n
<\/th>\nBalanced Accuracy<\/th>\nAUC<\/th>\nF1 Score<\/th>\nF1 Score Hold<\/th>\n<\/tr>\n<\/thead>\n
Turn Prediction v3<\/strong><\/td>\n88.05<\/strong><\/td>\n94.58<\/strong><\/td>\n84.44<\/td>\n91.20<\/td>\n<\/tr>\n
SmartTurn v3.2<\/td>\n77.41<\/td>\n88.81<\/td>\n70.88<\/td>\n86.44<\/td>\n<\/tr>\n
Deepgram Flux<\/td>\n87.10<\/td>\n\u2014<\/td>\n84.60<\/strong><\/td>\n92.60<\/strong><\/td>\n<\/tr>\n
LiveKit<\/td>\n82.70<\/td>\n88.70<\/td>\n76.70<\/td>\n83.30<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

Turn Prediction v3 leads on balanced accuracy and AUC across all conditions. Full benchmarks and our public test dataset are in the technical deep-dive<\/a>.<\/p>\n

Interruption Prediction v1: knowing when to stop<\/h3>\n

When you’re listening to someone and you say “yeah” or “okay” or “got it,” you’re not interrupting. You’re saying “I’m with you, keep going.” But when you say “wait, stop, that’s not what I meant,” you need the other person to actually stop.<\/p>\n

 <\/p>\n

Without interruption prediction, bots can’t tell the difference. Every sound the user makes while the bot is talking gets treated the same way. Either the bot stops on every “uh-huh,” which is annoying, or it plows through when someone actually needs to jump in, which is worse.<\/p>\n

 <\/p>\n

Interruption Prediction v1 is the first audio-only model in the industry built to solve this.<\/strong> It figures out whether the user actually wants to interrupt or is just giving feedback. From the audio alone, no transcription needed, without waiting for a full sentence. It reacts in under a second with less than 6% false positives. It handles laughter, coughing, and sneezing correctly too, with under 5% false triggers on non-speech sounds. The bot stops when you need it to, and keeps going when you don’t.<\/p>\n

 <\/p>\n

Turn Prediction and Interruption Prediction are two sides of the same coin. One reads the silence, the other reads the speech. Together, they give a voice agent something no reactive system has: the ability to read the room.<\/p>\n

Signal Detectors: a thousand tiny cues<\/h3>\n

We don’t just read conversational flow from someone’s voice. We pick up on who they are. Whether they’re a real person or a recording. Their gender, age group, accent. We do this without thinking, in milliseconds. Signal Detectors brings this to voice AI with a new set of small, real-time models launching with three:<\/p>\n