The recordings, transcripts, segment labels, metadata and every VI model's processed output are public. Re-run any number on this page.
Dataset
265 recordings, three scenarios
Recordings, transcripts, segment labels and metadata.
load_dataset("Krisp-AI/VoiceIsolation-Benchmark-Dataset")
View dataset on Hugging Face
Processed dataset
Every VI model's output on every file
Default, Balanced, Lite and HD outputs for all 265 recordings, ready to run through any STT.
load_dataset("Krisp-AI/VoiceIsolation-Benchmark-Dataset-Processed")
View processed dataset
Method
How the numbers are computed
WER with jiwer 4.0.0, word-level alignment, corpus-level pooling. Three-stage normalization on reference and hypothesis: structural regex splitting, NVIDIA NeMo WFST, then jiwer lowercase, contractions, punctuation and filler removal. Perceptual quality with DNSMOS-C (NISQA-trained) on primary-speaker segments only.
How transcripts are normalized before scoring
Reference and hypothesis go through the same three steps. First, a set of regex rules splits postcodes, letter-digit tokens like NG-228 or Flex30, standalone four-digit numbers (except likely years) and adjacent digit groups such as phone numbers. Then NVIDIA NeMo's WFST normalizer spells out numbers, currency, ordinals, times and abbreviations. Finally jiwer lowercases, expands contractions, strips punctuation, removes fillers such as um and uh, and aligns at word level. An engine is never penalized for writing 1928 instead of spelling it.
