{"id":23356,"date":"2026-06-09T16:53:44","date_gmt":"2026-06-09T12:53:44","guid":{"rendered":"https:\/\/krisp.ai\/blog\/?p=23356"},"modified":"2026-06-09T16:53:44","modified_gmt":"2026-06-09T12:53:44","slug":"introducing-voice-translation-api","status":"publish","type":"post","link":"https:\/\/krisp.ai\/blog\/introducing-voice-translation-api\/","title":{"rendered":"Introducing the Voice Translation API: Real-Time Speech-to-Speech Translation for Developers"},"content":{"rendered":"

The engine behind Krisp’s enterprise voice translation, with over 1M+ minutes of production call translation, tested across 30 languages, 6 business domains, and 870 real conversations, is now available as a self-serve API.<\/em><\/p>\n

The demo-to-production gap in voice translation<\/h2>\n

Getting a real-time voice translation demo working is easy. Getting it to survive production is the hard part.<\/p>\n

Real users have accents. They speak over background noise. They use domain-specific terms that carry the most weight in the conversation: medication names, policy numbers, account details, email addresses. These are exactly the terms that get hallucinated or garbled by general-purpose translation engines. And there’s no built-in feedback mechanism to tell you when it happens. Your first quality signal is a user complaint, a compliance flag, or a patient safety issue.<\/p>\n

Most voice translation APIs report accuracy on clean benchmark recordings made in studio conditions. Those numbers typically drop 5 to 10 points in production. The gap between what works in a demo and what works on a real call with a real customer is where most translation features fail.<\/p>\n

We built the Voice Translation API<\/a> to close that gap. Production-grade speech-to-speech translation, the same engine running in live enterprise contact centers today, now available self-serve.<\/p>\n

Don’t take our word for it. Try the playground \u2192<\/strong><\/a> Speak into it, pick a language pair, and hear the output yourself. No integration needed, no signup required.<\/p>\n

The engine: 96% accuracy on real calls, not studio audio<\/h2>\n

This is not a new engine. It is the same translation engine that powers Krisp Voice Translation in live enterprise contact centers, with over 1M+ minutes of production call translation. Same model, same accuracy, same language support.<\/p>\n

What makes this engine different from other voice translation APIs is where it was built and how accuracy was measured. Enterprise contact centers are the most unforgiving environment for voice AI. Frustrated customers speaking fast. Background noise from open floor plans. Heavy accents across dozens of languages. Account verifications where every digit matters. Calls where a translation error means a compliance violation, a disputed claim, or a patient safety incident.<\/p>\n

That pressure produced an engine with production data no benchmark can replicate:<\/p>\n

<\/p>\n\n\n\n\n\n\n\n\n\n
Metric<\/td>\nResult<\/td>\n<\/tr>\n
Translation accuracy<\/td>\n96% on live calls with real accents and noise (AutoQA-scored)<\/td>\n<\/tr>\n
Calls handled without interpreter<\/td>\n89% end-to-end<\/td>\n<\/tr>\n
Patient safety incidents<\/td>\nZero (across 8+ languages in a healthcare deployment)<\/td>\n<\/tr>\n
AHT reduction vs. human language services<\/td>\n20%+<\/td>\n<\/tr>\n
Interpreter wait time reduction<\/td>\nMore than 2x<\/td>\n<\/tr>\n
Production minutes translated<\/td>\n1M+<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

These numbers come from real calls with real consequences, not curated test sets. Most speech-to-speech translation APIs report accuracy on clean benchmark audio recorded in studio conditions. We measure where it counts.<\/p>\n

Benchmark data: 30 languages, 6 domains, 870 conversations<\/h2>\n

Beyond production deployments, the engine has been independently evaluated using three validation layers: automated benchmarking, AI-driven semantic scoring (AutoQA), and bilingual human review by professional linguists across 8 languages.<\/p>\n\n\n\n\n\n\n\n
Metric<\/td>\nResult<\/td>\n<\/tr>\n
English transcription accuracy (WER)<\/td>\n~2.7% (97 out of 100 words correct)<\/td>\n<\/tr>\n
Target language transcription accuracy<\/td>\n2\u201310% WER for most languages<\/td>\n<\/tr>\n
Translation quality (BLEU), top languages<\/td>\n51\u201366 (human translations typically score ~60)<\/td>\n<\/tr>\n
Semantic accuracy (AutoQA)<\/td>\n94\u201396 out of 100 across all 30 benchmarked languages<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

How we measured it<\/h2>\n

Transcription<\/strong> was measured using Word Error Rate (WER), the industry standard for speech recognition accuracy. Top languages like Italian (2.07%) and Spanish (2.11%) achieve WER under 2.5%.<\/p>\n

Translation<\/strong> was measured using BLEU, scored bidirectionally (English\u2192target and target\u2192English). We also used chrF++, a character-level metric that complements BLEU for morphologically complex languages like Turkish, Finnish, and Hungarian, where word-level BLEU alone can understate quality.<\/p>\n

AutoQA<\/strong>, Krisp’s semantic scoring system, independently validated every conversation across four dimensions: intent accuracy (35% weight), entity accuracy (30%), conversation flow (25%), and naturalness (10%). Scores averaged 94\u201396 across all 30 languages.<\/p>\n

Bilingual human review<\/strong> by professional linguists across 8 languages independently confirmed the automated findings.<\/p>\n

<\/p>\n

What the API does<\/h2>\n

Speech in one language, speech and text out in another. Real-time, synchronous, built for live conversations.<\/p>\n

Here’s the simplest integration. Configure a session, open it with callbacks, and stream audio:<\/p>\n

# Configure and open a session<\/code>
\nconfig = VtSessionConfig(<\/code>
\nauth_token = session_key,<\/code>
\ninput_language_code = \"en-US\",<\/code>
\noutput_language_code = \"es-US\",<\/code>
\nvoice = VtVoice.FEMALE,<\/code>
\n)<\/code><\/p>\n

vt = Vt.create(config,<\/code>
\naudio_result_callback = on_audio,<\/code>
\ntranslated_transcript_callback = on_text,<\/code>
\n)<\/code><\/p>\n

# Stream audio in, get speech + text out<\/code>
\nfor chunk in pcm_chunks:<\/code>
\nvt.process(chunk)<\/code><\/p>\n

That’s it. Speech in, translated speech and text out. Python and JavaScript SDKs ship with sample code and a quickstart guide. From zero to translated audio in 5 minutes.<\/p>\n

Here’s what the engine handles that most real-time translation APIs leave to you:<\/p>\n

Built-in Background Voice Cancellation.<\/strong> Background noise, competing voices, reverberation. The conditions that degrade translation quality in every real-world deployment are handled before translation begins. Configurable via the API. You don’t need clean audio input.<\/p>\n

Native accent robustness.<\/strong> Indian-accented English, Hispanic-accented English, regional accents across every supported language. Accuracy doesn’t degrade. The engine was built on the full spectrum of how people actually speak, not how they speak in recording studios.<\/p>\n

Accurate handling of names, numbers, and emails.<\/strong> Policy numbers, medication names, account details, email addresses, dates of birth. The kind of content that typically gets hallucinated or garbled comes through accurately.<\/p>\n

61 languages with any-to-any pairs.<\/strong> Not just “Spanish” but US Spanish, European Spanish, and the engine distinguishes between them. French Canadian and metropolitan French. Egyptian Arabic. Regional languages like Catalan, Galician, and Basque. The full list is available via the languages endpoint and updated dynamically.<\/p>\n

Real-time transcripts.<\/strong> Interim, final, and translated transcripts streamed alongside translated audio. Each independently toggleable via the session config.<\/p>\n

<\/p>\n

Under the hood: technical details<\/h2>\n

Authentication<\/h3>\n

Two-step authorization keeps your long-lived API key off the client-side connection:<\/p>\n

    \n
  1. API Key<\/strong> from the developer dashboard. Used to generate short-lived session keys.<\/li>\n
  2. Session Key<\/strong>, a temporary scoped token with configurable TTL (5 minutes to 24 hours). Passed as a query parameter when opening the WebSocket.<\/li>\n<\/ol>\n

    GET https:\/\/api.developers.krisp.ai\/v2\/sdk\/voice-translation\/session\/token?expiration_ttl=100<\/code>
    \nAuthorization: api-key API_KEY<\/code><\/p>\n

    The long-lived key never touches the WebSocket connection directly.<\/p>\n

    Session configuration<\/h3>\n

    Every session is controlled through a single JSON config message sent after the WebSocket connects. Source and target language, output voice, custom vocabulary, translation dictionary, transcript toggles, background voice cancellation, and client metadata are all set in one message:<\/p>\n

    {<\/code>
    \n\"config\": {<\/code>
    \n\"source_language\": \"en-US\",<\/code>
    \n\"target_language\": \"es-US\",<\/code>
    \n\"voice\": \"female\",<\/code><\/p>\n

    \"vocabulary\": [\"Lisinopril\", \"metformin\", \"HIPAA\"],<\/code>
    \n\"translation_dictionary\": [<\/code>
    \n{ \"source\": \"copay\", \"target\": \"copago\" },<\/code>
    \n{ \"source\": \"referral\", \"target\": \"remisi\u00f3n\" }<\/code>
    \n],<\/code><\/p>\n

    \"transcript\": {<\/code>
    \n\"interim\": true,<\/code>
    \n\"final\": true,<\/code>
    \n\"translate\": true<\/code>
    \n},<\/code><\/p>\n

    \"features\": {<\/code>
    \n\"background_voice_cancellation\": true<\/code>
    \n}<\/code>
    \n}<\/code>
    \n}<\/code><\/p>\n

    Domain customization from day one<\/h3>\n

    Custom Vocabulary<\/strong> improves transcription accuracy. Add terms the engine should recognize: product names, medical terminology, internal codes. If you’re in healthcare, you add your medication names. If you’re in insurance, you add your product terms.<\/p>\n

    Translation Dictionary<\/strong> controls how recognized terms are translated. Define specific source \u2192 target mappings per language pair. Map “copay” to “copago” in Spanish. Map “deductible” to “Selbstbehalt” in German. You control both recognition and translation output.<\/p>\n

    Both are configured per session via the JSON config. No training step, no fine-tuning, no waiting. Add your terms and they’re active immediately.<\/p>\n

    <\/p>\n

    Server events<\/h3>\n

    Three event types come back alongside translated audio frames:<\/p>\n