August 12, 2026

Voice Isolation 2.5: Built for STT, Not Just Human Ears

Written by Krisp Engineering Team

Max 9 min read

Share this post

Get Krisp for Free

VIVA is a collection of real-time AI models for voice agents, including Voice Isolation (VI), turn-taking, and interruption prediction. The Voice Isolation model comes first in the chain, delivering clean audio by removing noise and background speech so the turn-taking model can tell when a caller has actually finished speaking. Those models run in production today across more than 1B+ minutes of voice-AI conversations every month.

Having achieved effective voice isolation in the VIVA models, we saw customers running VIVA in front of their STT as well, improving transcription accuracy on real-world calls, where background speech like a TV playing in the room would otherwise end up in the transcript.

That points to something modern speech-to-text still gets wrong. Background noise robustness in modern STT systems is largely a solved problem: feed today’s STT models fan noise or traffic, and they transcribe it well. Other people talking in the background is not transcribed well. When a second voice overlaps the primary speaker, word error rates degrade, and conventional denoising can’t help, because the interference is speech, not noise. Separating the primary speaker from competing voices is exactly what VIVA’s Voice Isolation does.

And the payoff is large.

🗣️ On calls with competing speakers, putting VIVA in front of the STT cuts average WER (word error rate) from around 36% to 11% across the STT models we tested, cutting the errors by roughly 70%. That is something that simply noise filtering can’t solve.

But as more teams run isolation ahead of an STT, the feedback and example recordings customers shared. It led us to a consistent insight:

People and STT models don’t listen the same way.

To isolate a primary speaker from overlapping conversation, VI sometimes has to aggressively remove secondary speech, and on the hardest segments, that removal can affect the primary voice too. To a human listener, the result is still perfectly usable. But an STT model, which isn’t trained on isolated-voice output, can read those affected moments as noise and drop words that were actually spoken.

The result wasn’t noise in the transcript; it was gaps in it. Deletions.

VI 2.5, the Voice Isolation model within VIVA that we’re releasing today, is built to address that feedback. It isolates voice more surgically and shapes the output so STT models transcribe it more faithfully: fewer deletions on challenging segments and a lower WER across every model we tested. We’re shipping it in two sizes: the full VI 2.5, and a lite version for CPU-constrained and edge deployments.

Introducing Voice Isolation (VI) 2.5

VI 2.5 is our latest and most advanced general-purpose voice isolation model for conversational AI. It is STT-agnostic and lowers WER against no processing on every engine we tested, with its largest gains in reverberant conditions.

Four results define this release:

WER drops 46.4% on average. From 17.9% on untouched audio to 10.2% with Voice Isolation
2.5 in front of the STT, and the gain holds on every engine we tested.

69.7% fewer errors on competing speech. On calls with a competing speaker, the case that
decides whether a real call works, word error rate falls from 37% to 10.8% versus no
processing, the interference denoising can’t touch.

The clean-audio penalty is nearly gone. With nothing to remove, the only way the model can move WER is by touching the primary voice. Earlier models added errors here; VI 2.5 pulls clean-audio WER back to 2.15%, essentially even with no processing (2.10%). You can leave isolation on for every call.

A lite model ~3.5× smaller. VI 2.5 lite needs far less compute and, on typical calls, is level with the full model. If CPU per stream is what has kept isolation off most of your traffic, this is the version that removes the objection.

We evaluated on 1,685 real-world recordings, about 7 hours of audio, grouped into three conditions, each transcribed by all 10 STT systems.

How Voice Isolation 2.5 lowers Word Error Rate

Across conditions, VI 2.5 delivers a 46.4% average reduction in WER versus unprocessed audio (15.35% → 8.22%). Below is the average WER by condition, across 10 STT systems from seven leading vendors:

We didn’t tune for a single STT model. Every number below is averaged across 10 streaming and non-streaming STT systems from seven vendors: Deepgram, NVIDIA, Soniox, ElevenLabs, AssemblyAI, Google, and Cartesia. Here, WER is averaged across all three test conditions.

Looking further into the results, two cases stood out.

VI 2.5 is far gentler in challenging acoustic conditions

Complex acoustic environments, whether severe room echo, heavy overlapping noise, or unpredictable acoustic dynamics, may cause models to apply overly aggressive filtering. When difficult acoustic cues confuse the isolation model, excessive processing creates subtle artifacts that impair STT performance, resulting in a higher WER than raw audio. VI 2.5 introduces a more nuanced awareness of these tricky scenarios to avoid over-processing. Consequently, on these challenging recordings, WER dropped from v2.1’s 15.42% to 11.61%. While still slightly above the unprocessed baseline of 8.02%, VI 2.5 represents a major step forward in closing the gap on scenarios where earlier models over-filtered the signal.

VI 2.5 nearly eliminates the clean-audio penalty.

This is the case that matters most for trust: with no competing speaker to remove, any change in WER is the model touching the primary voice. Earlier models added errors here (v2.1 sat at 2.24% versus 2.10% with no processing at all), the reason some teams saw transcription get worse on quiet audio.

VI 2.5 pulls that back to 2.15%, essentially even with no processing and an improvement on v2.1 on 8 of the 10 systems we tested. Leaving isolation on now costs almost nothing on clean calls.

Voice Isolation 2.5 Lite: same gains, smaller model

Not every deployment can spare the full model’s footprint. For CPU-constrained and edge scenarios, we’ve brought the same upgrades to our lite voice isolation model, a much smaller model that now matches the full one.

Averaged across the same test set (10 STT systems), VI lite 2.5 cuts overall WER from 12.45% (lite v1) to 8.23%, matching full VI 2.5.1 (8.22%) on CPU at roughly a third of the size. The biggest jump is in reverberant rooms, where the previous lite model struggled most: WER drops from 20.55% to 10.55%.

Specs and integration

VI (Voice Isolation) 2.5 is a real-time model that fits server-side voice AI pipelines. VI lite 2.5 matches the full model on specs and integration at roughly 3.5× smaller parameter size.

Both models stay robust on Bluetooth headsets like AirPods, where the microphone sits well away from the speaker’s mouth, a placement that normally degrades isolation.

VIVA sits at the front of your pipeline, before speech-to-text, so the STT, and every component after it, sees a clean primary-speaker signal. Integration is unchanged from the current VIVA SDK.

What’s next

VIVA 2.5 largely solved turn-taking for voice agents, giving them clean speech to work with by isolating the primary speaker from background noise and competing voices. VI (Voice Isolation) 2.5 extends that work to the transcript itself: by shaping isolated speech that STT models read faithfully, it lowers word error rate across every engine we tested and removes the deletions that used to slip into transcripts on the hardest segments. For a voice agent, that means fewer misheard words, fewer wrong turns, and more dependable behavior downstream.

We’re not done:

• Sharper isolation where there’s least to remove. We’re pushing voice isolation further
  on competing-speaker and clean audio: the conditions where the primary voice most needs
  protecting.
• Open benchmarks. We plan to open-source much of the benchmark data behind these
  results, so teams can reproduce them and evaluate their own pipelines.

FAQs