{"id":9654,"date":"2022-09-23T16:47:00","date_gmt":"2022-09-23T12:47:00","guid":{"rendered":"https:\/\/krisp.ai\/blog\/?p=9654"},"modified":"2023-07-23T13:10:58","modified_gmt":"2023-07-23T09:10:58","slug":"noise-cancellation-quality-evaluation","status":"publish","type":"post","link":"https:\/\/krisp.ai\/blog\/noise-cancellation-quality-evaluation\/","title":{"rendered":"Noise Cancellation Quality Evaluation: How We Test Krisp\u2019s Technology"},"content":{"rendered":"<p><span style=\"font-weight: 400;\">The power of Krisp lies in its AI-based noise-cancellation algorithm\u2014which we are constantly perfecting. Noise cancellation comes with distinct technical challenges: Namely, how do we remove distracting background noise while preserving human voice? If the app overcorrects on noise removal, the speaker\u2019s voice will sound robotic. If it overcorrects on voice preservation, unwanted sounds will remain.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">In each node of the audio pipeline, there are standardized methods for measuring the quality. In this blog, we\u2019re going to deep dive into the nuances and challenges of software-based real-time working Noise Cancellation (NC) quality evaluations. We\u2019ll show you how we do algorithm testing here at Krisp. But first, let\u2019s begin with the basics.<\/span><\/p>\n<h2><span style=\"font-weight: 400;\">What affects sound quality?<\/span><\/h2>\n<h3><span style=\"font-weight: 400;\">Our perception<\/span><\/h3>\n<p><span style=\"font-weight: 400;\">Hearing is a <\/span><span style=\"font-weight: 400;\">physiological process<\/span><span style=\"font-weight: 400;\"> and is based on human perception. Sound waves reach your ears and create an auditory perception that varies depending on the medium through which it traveled and the way your particular ears are structured. For example, you will perceive the same sound differently in the air versus in water. Another example is computer-generated audio that some people hear as \u201cyanny\u201d and others as \u201claurel.\u201d Check it out below. What do you hear?<\/span><\/p>\n<p><iframe title=\"YouTube video player\" src=\"https:\/\/www.youtube.com\/embed\/7X_WvGAhMlQ\" width=\"630\" height=\"315\" frameborder=\"0\" allowfullscreen=\"allowfullscreen\"><\/iframe><\/p>\n<p><span style=\"font-weight: 400;\">So, sound quality is based on our perception, which leads to a strong subjectivity factor.\u00a0<\/span><\/p>\n<h3><span style=\"font-weight: 400;\">Microphones and speakers<\/span><\/h3>\n<p><span style=\"font-weight: 400;\">Another factor affecting overall sound quality is the audio recording and reproduction device used. Microphones can introduce specific audible and inaudible distortions like clipping, choppy voice, or suppressed frequency ranges. Moreover, they capture background noises and voices along with the main speaker\u2019s voice. Though there are devices that have built-in noise cancellation, these kinds of hardware solutions are often not affordable for everyone or don\u2019t eliminate the background noise completely.\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400;\">In contrast, Krisp provides an AI-based software solution that increases the call quality with any device by effectively identifying and removing unwanted sounds. Developing and perfecting this app has unique challenges. Let\u2019s dive into how we at Krisp test our noise-cancellation algorithm.<\/span><\/p>\n<h2><span style=\"font-weight: 400;\">3 testing challenges of noise-cancellation algorithms<\/span><\/h2>\n<h3><span style=\"font-weight: 400;\">1. AI-based system testing is different from non-AI-based system testing.<\/span><\/h3>\n<p><span style=\"font-weight: 400;\">The first challenge of NC algorithm quality assurance is that we\u2019re dealing with an <\/span><b>AI-based system<\/b><span style=\"font-weight: 400;\">. AI testing, in general, is different from non-AI-based software testing. In the case of <\/span><b>non-AI-based software testing<\/b><span style=\"font-weight: 400;\">, the test subject is the predefined <\/span><b>desired<\/b> <b>behavior<\/b><span style=\"font-weight: 400;\">. In the case of <\/span><b>AI testing<\/b><span style=\"font-weight: 400;\">, however, the test subject is the <\/span><b>logic<\/b><span style=\"font-weight: 400;\"> of the system. See Figure 1 below.<\/span><\/p>\n<p><img loading=\"lazy\" class=\"alignnone size-full wp-image-9655\" src=\"https:\/\/krisp.ai\/blog\/wp-content\/uploads\/2022\/09\/Testing-flow-of-Software-and-AI-systems.png\" alt=\"Testing flow\" width=\"1242\" height=\"540\" srcset=\"https:\/\/krisp.ai\/blog\/wp-content\/uploads\/2022\/09\/Testing-flow-of-Software-and-AI-systems.png 1242w, https:\/\/krisp.ai\/blog\/wp-content\/uploads\/2022\/09\/Testing-flow-of-Software-and-AI-systems-300x130.png 300w, https:\/\/krisp.ai\/blog\/wp-content\/uploads\/2022\/09\/Testing-flow-of-Software-and-AI-systems-380x165.png 380w, https:\/\/krisp.ai\/blog\/wp-content\/uploads\/2022\/09\/Testing-flow-of-Software-and-AI-systems-768x334.png 768w\" sizes=\"(max-width: 1242px) 100vw, 1242px\" \/><\/p>\n<p><i><span style=\"font-weight: 400;\">Figure 1: Testing flow of Software and AI systems<\/span><\/i><\/p>\n<h3><span style=\"font-weight: 400;\">2. Noise cancellation has tradeoffs.<\/span><\/h3>\n<p><span style=\"font-weight: 400;\">The second challenge is that we always need to consider the possible <\/span><b>tradeoffs<\/b><span style=\"font-weight: 400;\"> of the system.\u00a0<\/span><\/p>\n<p><b>NC aggressiveness<\/b><\/p>\n<p><span style=\"font-weight: 400;\">One such tradeoff is the NC aggressiveness, which is the level of noise cancellation we apply. If the level is too low, distracting sounds still come through. If the level is too high, the speaker\u2019s voice sounds robotic. The NC algorithm must find the golden mean where it eliminates all of the background noises while preserving the speaker&#8217;s voice.<\/span><\/p>\n<p><b>Resource usage vs. quality<\/b><\/p>\n<p><span style=\"font-weight: 400;\">Another tradeoff is between resource usage and quality. Normally, the more extensive the neural network, the better work it will do. But the machines that are meant to run real-time working NC algorithms are supposed to have limited resources. So we need to assess consumed resources, like CPU usage, memory allocation, and power consumption to verify it\u2019s working on-device with the expected quality. In addition, <\/span><a href=\"https:\/\/krisp.ai\/blog\/voice-communication-quality-with-krisp-sdk\/\"><span style=\"font-weight: 400;\">SDKs<\/span><\/a><span style=\"font-weight: 400;\"> doing real-time noise cancellation need to be verified on multiple platforms, such as Mac, Windows, and iOS.\u00a0<\/span><\/p>\n<h3><span style=\"font-weight: 400;\">3. Audio quality can be affected by microphones and at multiple points along the journey.<\/span><\/h3>\n<p><span style=\"font-weight: 400;\">The third challenge of NC algorithm quality assurance is that microphone devices can add their own distortions. Plus, there are other factors impacting the audio quality as well. Before reaching the other side of the conferencing call, audio makes a long journey (see Figure 2). Each step could add its own degradations. A network can cause packet loss and other artifacts, which may lead to degradations in the output signal. On top of that, the sound-amplifying device in the final point should be qualified enough to replicate the audio with high fidelity.<\/span><\/p>\n<p><img loading=\"lazy\" class=\"alignnone size-full wp-image-9656\" src=\"https:\/\/krisp.ai\/blog\/wp-content\/uploads\/2022\/09\/Audio-Signal-Processing-Pipeline.png\" alt=\"Audio Signal Processing Pipeline\" width=\"624\" height=\"235\" srcset=\"https:\/\/krisp.ai\/blog\/wp-content\/uploads\/2022\/09\/Audio-Signal-Processing-Pipeline.png 624w, https:\/\/krisp.ai\/blog\/wp-content\/uploads\/2022\/09\/Audio-Signal-Processing-Pipeline-300x113.png 300w, https:\/\/krisp.ai\/blog\/wp-content\/uploads\/2022\/09\/Audio-Signal-Processing-Pipeline-380x143.png 380w\" sizes=\"(max-width: 624px) 100vw, 624px\" \/><\/p>\n<p><i><span style=\"font-weight: 400;\">Figure 2: Audio Signal Processing Pipeline<\/span><\/i><\/p>\n<h2><span style=\"font-weight: 400;\">Speech quality testing: Subjective evaluations<\/span><\/h2>\n<p><span style=\"font-weight: 400;\">The straightforward approach to Noise Cancellation quality evaluation is to listen to its output\u2014in technical terms, conduct subjective evaluations. People of different ages, ear structures, or even having different moods may perceive the same piece of audio differently. No doubt, this leads to a subjective bias. To decrease that bias as much as possible, the standard process for subjective tests has been introduced.\u00a0<\/span><\/p>\n<p><a href=\"https:\/\/www.itu.int\/dms_pubrec\/itu-r\/rec\/bs\/R-REC-BS.1284-2-201901-I!!PDF-E.pdf\" target=\"_blank\" rel=\"nofollow noopener\"><span style=\"font-weight: 400;\">ITU-R BS.1284-2<\/span><\/a><span style=\"font-weight: 400;\"> establishes the recommended standardized setup for conducting subjective tests. To get accurate results, you need to have diverse listeners as much as possible. More opinions, lower bias. The listeners rate the quality of audio on a scale of 1 to 5, where 5 corresponds to the highest possible quality and 1 to the lowest.\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400;\">In our case, the score represents a healthy average of intelligibility of the voice and audibility of residual noise (if any). The mean opinion score (MOS) comes from the arithmetic mean of all of the listeners\u2019 scores.\u00a0<\/span><\/p>\n<table role=\"presentation\">\n<tbody>\n<tr>\n<th><b>Quality of the Audio<\/b><\/th>\n<th><b>MOS Score<\/b><\/th>\n<\/tr>\n<tr>\n<td><span style=\"font-weight: 400;\">Excellent<\/span><\/td>\n<td><span style=\"font-weight: 400;\">5<\/span><\/td>\n<\/tr>\n<tr>\n<td><span style=\"font-weight: 400;\">Good<\/span><\/td>\n<td><span style=\"font-weight: 400;\">4<\/span><\/td>\n<\/tr>\n<tr>\n<td><span style=\"font-weight: 400;\">Fair<\/span><\/td>\n<td><span style=\"font-weight: 400;\">3<\/span><\/td>\n<\/tr>\n<tr>\n<td><span style=\"font-weight: 400;\">Poor<\/span><\/td>\n<td><span style=\"font-weight: 400;\">2<\/span><\/td>\n<\/tr>\n<tr>\n<td><span style=\"font-weight: 400;\">Bad<\/span><\/td>\n<td><span style=\"font-weight: 400;\">1<\/span><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p><i><span style=\"font-weight: 400;\">Figure 3: MOS score rating description<\/span><\/i><\/p>\n<h2><span style=\"font-weight: 400;\">Speech quality testing: Objective evaluations<\/span><\/h2>\n<p><span style=\"font-weight: 400;\">Though subjective tests are precise, they can be very costly and time-consuming, as they require a lot of human, time, and financial resources. Consider a situation having many variants of an NC algorithm with two or more NC algorithms under test. Conducting subjective evaluations for the algorithms may delay the early feedback, making it impossible to deliver the algorithms promptly.\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400;\">To overcome these issues, we\u2019re also considering objective evaluation metrics. Some of these metrics tend to be highly correlated with subjective scores. Unlike subjective scores, objective evaluations are repeatable. No matter how many times you evaluate the same algorithm, you will get the same scores\u2014something that is not guaranteed with subjective metrics. Objective evaluations make it possible to evaluate many NC algorithms in the same conditions without wasting extra effort and time.\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400;\">There are a few objective evaluation metrics designed for different use cases. Each has its own logic for audio assessment. Let&#8217;s review some of the metrics that we\u2019re currently considering.\u00a0<\/span><\/p>\n<p><b>PESQ<\/b><span style=\"font-weight: 400;\"> (Perceptual Evaluation of Speech Quality) is the most used metric in Research, though it&#8217;s not for NC quality evaluations explicitly, but for measuring speech quality after passing through the network and codec-related distortions. It is standardized as Recommendation <\/span><a href=\"https:\/\/www.itu.int\/rec\/T-REC-P.862\" target=\"_blank\" rel=\"nofollow noopener\"><span style=\"font-weight: 400;\">ITU-T P.862<\/span><\/a><span style=\"font-weight: 400;\">. Its result represents the mean opinion score (MOS) that covers a scale from 1 (bad) to 5 (excellent).\u00a0<\/span><\/p>\n<p><b>POLQA <\/b><span style=\"font-weight: 400;\">(Perceptual Objective Listening Quality Analysis) is an upgraded version of PESQ that provides an advanced level of benchmarking accuracy and adds significant new capabilities for super-wideband (HD) and full-band speech signals. It\u2019s standardized as Recommendation <\/span><a href=\"https:\/\/www.itu.int\/rec\/T-REC-P.863\" target=\"_blank\" rel=\"nofollow noopener\"><span style=\"font-weight: 400;\">ITU-T P.863<\/span><\/a><span style=\"font-weight: 400;\">. POLQA has the same MOS scoring scale as its predecessor PESQ, though it&#8217;s for the same use case of evaluating quality related to codec distortions.<\/span><\/p>\n<p><b>3QUEST <\/b><span style=\"font-weight: 400;\">(3-fold Quality Evaluation of Speech in Telecommunications) was designed to assess the background noise separately in a transmitted signal. Thus, it returns Speech-MOS (S-MOS), Noise-MOS (N-MOS), and General-MOS (G-MOS) values on a scale of 1 (bad) to 5 (excellent). G-MOS is a weighted average of the other two. It\u2019s standardized as Recommendation <\/span><a href=\"https:\/\/www.itu.int\/rec\/T-REC-P.835\" target=\"_blank\" rel=\"nofollow noopener\"><span style=\"font-weight: 400;\">ITU-T P.835<\/span><\/a><span style=\"font-weight: 400;\">. <\/span><b>Based on our experience, 3QUEST is the most suitable objective metric for NC evaluations.<\/b><\/p>\n<p><b>NISQA<\/b><span style=\"font-weight: 400;\"> (Non-intrusive Objective Speech Quality Assessment) is a relatively new metric. Unlike the above-listed metrics, it doesn\u2019t require the reference clean speech audio file. It\u2019s standardized from <\/span><a href=\"https:\/\/www.itu.int\/rec\/T-REC-P.800\" target=\"_blank\" rel=\"nofollow noopener\"><span style=\"font-weight: 400;\">ITU-T Rec. P.800 series<\/span><\/a><span style=\"font-weight: 400;\">. <\/span><span style=\"font-weight: 400;\">Besides MOS score, NISQA p<\/span><span style=\"font-weight: 400;\">rovides a prediction of the four speech quality dimensions: Noisiness, Coloration, Discontinuity, and Loudness.\u00a0<\/span><\/p>\n<h2><span style=\"font-weight: 400;\">How we generate test datasets<\/span><\/h2>\n<p><span style=\"font-weight: 400;\">A straightforward approach to generating test datasets is to mix the clean voice and noise with the desired SNR (Signal-to-Noise Ratio) using an audio editor or other tools. Though these recordings don\u2019t simulate the real use case and are artificial, we use such recordings in the initial testing phases as they are easy to collect and can catch obvious quality issues in the early stages of development.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">To have more accurate recordings for qualitative tests and evaluations, we\u2019re collaborating with well-equipped audio labs to get recordings with predefined use cases. A real use case is being simulated in ETSI rooms and being recorded with high-quality microphones, where:\u00a0<\/span><\/p>\n<ul>\n<li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\">The spoken sentences are phonetically balanced, intended to cover all possible sounds in a certain language.<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\">There are a few speakers in the same recordings, intended to cover speaker-independency tests.<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\">Recordings are done with different languages, intended to cover language independency tests.<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\">They are simulating various noises with different noise levels and recording them along with voice with the same mic.<\/span><\/li>\n<\/ul>\n<p><span style=\"font-weight: 400;\">As you can see, a lot of use cases are covered with the above-mentioned datasets. However, to ensure the Noise Cancellation algorithmic quality, we still need to consider some test scenarios that are outside of these recordings&#8217; scope. To fill that gap, we also perform in-house recordings.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">To ensure Krisp works with any device, we need to test it with a lot of devices. However, testing with <\/span><i><span style=\"font-weight: 400;\">all<\/span><\/i><span style=\"font-weight: 400;\"> possible devices is practically impossible and may not be necessary. Instead, we\u2019ve identified the top devices used by our users and targeted the testing on those devices. Currently, we have an in-house \u201cdevice farm\u201d with almost 50 mics, and we\u2019re continuously adding the latest available mics.\u00a0<\/span><\/p>\n<p><img loading=\"lazy\" class=\"alignnone size-full wp-image-9658\" src=\"https:\/\/krisp.ai\/blog\/wp-content\/uploads\/2022\/09\/test-set-microphones.png\" alt=\"test set microphones\" width=\"1920\" height=\"1080\" srcset=\"https:\/\/krisp.ai\/blog\/wp-content\/uploads\/2022\/09\/test-set-microphones.png 1920w, https:\/\/krisp.ai\/blog\/wp-content\/uploads\/2022\/09\/test-set-microphones-300x169.png 300w, https:\/\/krisp.ai\/blog\/wp-content\/uploads\/2022\/09\/test-set-microphones-380x214.png 380w, https:\/\/krisp.ai\/blog\/wp-content\/uploads\/2022\/09\/test-set-microphones-768x432.png 768w, https:\/\/krisp.ai\/blog\/wp-content\/uploads\/2022\/09\/test-set-microphones-1536x864.png 1536w\" sizes=\"(max-width: 1920px) 100vw, 1920px\" \/><\/p>\n<p><i><span style=\"font-weight: 400;\">Figure 4: Some microphones from our test set<\/span><\/i><\/p>\n<p><span style=\"font-weight: 400;\">As a part of the NC algorithm, Krisp also removes the room echo, or in more technical terms, reverberation. Hence we\u2019re considering a lot of rooms with different acoustic setups to guarantee meaningful coverage of reverberant cases.\u00a0<\/span><\/p>\n<h2><span style=\"font-weight: 400;\">Noise cancellation and speech quality: The eternal tradeoff\u00a0<\/span><\/h2>\n<p><span style=\"font-weight: 400;\">Again, when evaluating the quality of Krisp\u2019s NC algorithm, we must strike the right balance between removing background noise and preserving the speaker\u2019s voice. There is an eternal tradeoff between these two criteria.\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400;\">To gain a complete picture, we\u2019re considering different test scenarios\/datasets along with corresponding applicable metrics. There is no single metric reflecting the best representative value for the quality assessment. Each evaluation metric is designed for certain use cases; they are complementing each other rather than replacing. <\/span><span style=\"font-weight: 400;\">Testing an NC algorithm with several evaluation metrics<\/span><span style=\"font-weight: 400;\"> allows us to have a more comprehensive picture to assess the audio quality from different standpoints.\u00a0<\/span><\/p>\n<h2><span style=\"font-weight: 400;\">Try next-level audio and voice technologies \u00a0<\/span><\/h2>\n<p><a href=\"https:\/\/krisp.ai\/blog\/voice-communication-quality-with-krisp-sdk\/\" target=\"_blank\" rel=\"noopener\">Krisp licenses its SDKs<\/a>\u00a0to embed directly into applications and devices. <a href=\"https:\/\/krisp.ai\/developers\/\" target=\"_blank\" rel=\"noopener\">Learn more about Krisp&#8217;s SDKs<\/a> and begin your evaluation today.<\/p>\n<p><a href=\"https:\/\/krisp.ai\/developers\/\" target=\"_blank\" rel=\"noopener\"><img loading=\"lazy\" class=\"alignnone size-full wp-image-9589\" src=\"https:\/\/krisp.ai\/blog\/wp-content\/uploads\/2022\/09\/engineering-blog-cta.png\" alt=\"krisp sdk\" width=\"1280\" height=\"720\" srcset=\"https:\/\/krisp.ai\/blog\/wp-content\/uploads\/2022\/09\/engineering-blog-cta.png 1280w, https:\/\/krisp.ai\/blog\/wp-content\/uploads\/2022\/09\/engineering-blog-cta-300x169.png 300w, https:\/\/krisp.ai\/blog\/wp-content\/uploads\/2022\/09\/engineering-blog-cta-380x214.png 380w, https:\/\/krisp.ai\/blog\/wp-content\/uploads\/2022\/09\/engineering-blog-cta-768x432.png 768w\" sizes=\"(max-width: 1280px) 100vw, 1280px\" \/><\/a><\/p>\n<h3><span style=\"font-weight: 400;\">Sources:<\/span><\/h3>\n<ul>\n<li style=\"font-weight: 400;\" aria-level=\"1\"><a href=\"https:\/\/www.nidcd.nih.gov\/health\/how-do-we-hear#:~:text=Sound%20waves%20enter%20the%20outer,bones%20in%20the%20middle%20ear\" target=\"_blank\" rel=\"noopener\"><span style=\"font-weight: 400;\">How Do We Hear? | NIDCD<\/span><\/a><span style=\"font-weight: 400;\">.<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"1\"><a href=\"https:\/\/arxiv.org\/abs\/2203.16032\" target=\"_blank\" rel=\"nofollow noopener\"><span style=\"font-weight: 400;\">[2203.16032] ConferencingSpeech 2022 Challenge: Non-intrusive Objective Speech Quality Assessment (NISQA) Challenge for Online Conferencing Applications<\/span><\/a><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"1\"><a href=\"https:\/\/www.isca-speech.org\/archive\/pdfs\/interspeech_2021\/mittag21_interspeech.pdf\" target=\"_blank\" rel=\"nofollow noopener\"><span style=\"font-weight: 400;\">NISQA: A Deep CNN-Self-Attention Model for Multidimensional Speech Quality Prediction with Crowdsourced Datasets<\/span><\/a><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"1\"><a href=\"https:\/\/ecs.utdallas.edu\/loizou\/cimplants\/quality_assessment_chapter.pdf\"><span style=\"font-weight: 400;\">Speech Quality Assessment<\/span><\/a><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"1\"><a href=\"https:\/\/cdn.head-acoustics.com\/fileadmin\/data\/global\/Application-Notes\/Telecom\/3QUEST-Application-Note.pdf\" target=\"_blank\" rel=\"nofollow noopener\"><span style=\"font-weight: 400;\">3QUEST: 3-fold Quality Evaluation of Speech in Telecommuni- cations<\/span><\/a><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"1\"><a href=\"https:\/\/www.itu.int\/dms_pubrec\/itu-r\/rec\/bs\/R-REC-BS.1284-2-201901-I!!PDF-E.pdf\" target=\"_blank\" rel=\"nofollow noopener\"><span style=\"font-weight: 400;\">General methods for the subjective assessment of sound quality<\/span><\/a><\/li>\n<\/ul>\n<hr \/>\n<p>This article is written by:<\/p>\n<p>Ani Chilingaryan, MS in Computer Science, CTFL | QA Manager, Research<\/p>\n","protected":false},"excerpt":{"rendered":"<p>The power of Krisp lies in its AI-based noise-cancellation algorithm\u2014which we are constantly perfecting. Noise cancellation comes with distinct technical challenges: Namely, how do we remove distracting background noise while preserving human voice? If the app overcorrects on noise removal, the speaker\u2019s voice will sound robotic. If it overcorrects on voice preservation, unwanted sounds will [&hellip;]<\/p>\n","protected":false},"author":65,"featured_media":9661,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"two_page_speed":[]},"categories":[421],"tags":[],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v24.2 (Yoast SEO v23.6) - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Noise Cancellation Quality Evaluation: How We Test Krisp\u2019s Technology<\/title>\n<meta name=\"description\" content=\"Peek inside the Krisp Research Team\u2019s noise cancellation quality evaluation. We\u2019ll show you how we conduct speech quality testing on our NC algorithm.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/krisp.ai\/blog\/noise-cancellation-quality-evaluation\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Noise Cancellation Quality Evaluation: How We Test Krisp\u2019s Technology\" \/>\n<meta property=\"og:description\" content=\"Peek inside the Krisp Research Team\u2019s noise cancellation quality evaluation. We\u2019ll show you how we conduct speech quality testing on our NC algorithm.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/krisp.ai\/blog\/noise-cancellation-quality-evaluation\/\" \/>\n<meta property=\"og:site_name\" content=\"Krisp\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/krispHQ\/\" \/>\n<meta property=\"article:published_time\" content=\"2022-09-23T12:47:00+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2023-07-23T09:10:58+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/krisp.ai\/blog\/wp-content\/uploads\/2022\/09\/Visulas-52.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1000\" \/>\n\t<meta property=\"og:image:height\" content=\"700\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"Krisp Research Team\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@krispHQ\" \/>\n<meta name=\"twitter:site\" content=\"@krispHQ\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/krisp.ai\/blog\/noise-cancellation-quality-evaluation\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/krisp.ai\/blog\/noise-cancellation-quality-evaluation\/\"},\"author\":{\"name\":\"Krisp Research Team\",\"@id\":\"https:\/\/krisp.ai\/blog\/#\/schema\/person\/172d23b73915155e0ab4e97868216bd1\"},\"headline\":\"Noise Cancellation Quality Evaluation: How We Test Krisp\u2019s Technology\",\"datePublished\":\"2022-09-23T12:47:00+00:00\",\"dateModified\":\"2023-07-23T09:10:58+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/krisp.ai\/blog\/noise-cancellation-quality-evaluation\/\"},\"wordCount\":1836,\"commentCount\":4,\"publisher\":{\"@id\":\"https:\/\/krisp.ai\/blog\/#organization\"},\"image\":{\"@id\":\"https:\/\/krisp.ai\/blog\/noise-cancellation-quality-evaluation\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/krisp.ai\/blog\/wp-content\/uploads\/2022\/09\/Visulas-52.png\",\"articleSection\":[\"Engineering Blog\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/krisp.ai\/blog\/noise-cancellation-quality-evaluation\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/krisp.ai\/blog\/noise-cancellation-quality-evaluation\/\",\"url\":\"https:\/\/krisp.ai\/blog\/noise-cancellation-quality-evaluation\/\",\"name\":\"Noise Cancellation Quality Evaluation: How We Test Krisp\u2019s Technology\",\"isPartOf\":{\"@id\":\"https:\/\/krisp.ai\/blog\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/krisp.ai\/blog\/noise-cancellation-quality-evaluation\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/krisp.ai\/blog\/noise-cancellation-quality-evaluation\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/krisp.ai\/blog\/wp-content\/uploads\/2022\/09\/Visulas-52.png\",\"datePublished\":\"2022-09-23T12:47:00+00:00\",\"dateModified\":\"2023-07-23T09:10:58+00:00\",\"description\":\"Peek inside the Krisp Research Team\u2019s noise cancellation quality evaluation. We\u2019ll show you how we conduct speech quality testing on our NC algorithm.\",\"breadcrumb\":{\"@id\":\"https:\/\/krisp.ai\/blog\/noise-cancellation-quality-evaluation\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/krisp.ai\/blog\/noise-cancellation-quality-evaluation\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/krisp.ai\/blog\/noise-cancellation-quality-evaluation\/#primaryimage\",\"url\":\"https:\/\/krisp.ai\/blog\/wp-content\/uploads\/2022\/09\/Visulas-52.png\",\"contentUrl\":\"https:\/\/krisp.ai\/blog\/wp-content\/uploads\/2022\/09\/Visulas-52.png\",\"width\":1000,\"height\":700,\"caption\":\"noise cancellation quality evaluation\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/krisp.ai\/blog\/noise-cancellation-quality-evaluation\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/krisp.ai\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Noise Cancellation Quality Evaluation: How We Test Krisp\u2019s Technology\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/krisp.ai\/blog\/#website\",\"url\":\"https:\/\/krisp.ai\/blog\/\",\"name\":\"Krisp\",\"description\":\"Blog\",\"publisher\":{\"@id\":\"https:\/\/krisp.ai\/blog\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/krisp.ai\/blog\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/krisp.ai\/blog\/#organization\",\"name\":\"Krisp\",\"url\":\"https:\/\/krisp.ai\/blog\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/krisp.ai\/blog\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/krisp.ai\/blog\/wp-content\/uploads\/2024\/10\/K.png\",\"contentUrl\":\"https:\/\/krisp.ai\/blog\/wp-content\/uploads\/2024\/10\/K.png\",\"width\":696,\"height\":696,\"caption\":\"Krisp\"},\"image\":{\"@id\":\"https:\/\/krisp.ai\/blog\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/www.facebook.com\/krispHQ\/\",\"https:\/\/x.com\/krispHQ\",\"https:\/\/www.linkedin.com\/company\/krisphq\/\",\"https:\/\/www.youtube.com\/channel\/UCAMZinJdR9P33fZUNpuxXtg\"]},{\"@type\":\"Person\",\"@id\":\"https:\/\/krisp.ai\/blog\/#\/schema\/person\/172d23b73915155e0ab4e97868216bd1\",\"name\":\"Krisp Research Team\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/krisp.ai\/blog\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/49fc839d54b3ccba70e28ccaad1472a7?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/49fc839d54b3ccba70e28ccaad1472a7?s=96&d=mm&r=g\",\"caption\":\"Krisp Research Team\"},\"url\":\"https:\/\/krisp.ai\/blog\/author\/research-team\/\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Noise Cancellation Quality Evaluation: How We Test Krisp\u2019s Technology","description":"Peek inside the Krisp Research Team\u2019s noise cancellation quality evaluation. We\u2019ll show you how we conduct speech quality testing on our NC algorithm.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/krisp.ai\/blog\/noise-cancellation-quality-evaluation\/","og_locale":"en_US","og_type":"article","og_title":"Noise Cancellation Quality Evaluation: How We Test Krisp\u2019s Technology","og_description":"Peek inside the Krisp Research Team\u2019s noise cancellation quality evaluation. We\u2019ll show you how we conduct speech quality testing on our NC algorithm.","og_url":"https:\/\/krisp.ai\/blog\/noise-cancellation-quality-evaluation\/","og_site_name":"Krisp","article_publisher":"https:\/\/www.facebook.com\/krispHQ\/","article_published_time":"2022-09-23T12:47:00+00:00","article_modified_time":"2023-07-23T09:10:58+00:00","og_image":[{"width":1000,"height":700,"url":"https:\/\/krisp.ai\/blog\/wp-content\/uploads\/2022\/09\/Visulas-52.png","type":"image\/png"}],"author":"Krisp Research Team","twitter_card":"summary_large_image","twitter_creator":"@krispHQ","twitter_site":"@krispHQ","schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/krisp.ai\/blog\/noise-cancellation-quality-evaluation\/#article","isPartOf":{"@id":"https:\/\/krisp.ai\/blog\/noise-cancellation-quality-evaluation\/"},"author":{"name":"Krisp Research Team","@id":"https:\/\/krisp.ai\/blog\/#\/schema\/person\/172d23b73915155e0ab4e97868216bd1"},"headline":"Noise Cancellation Quality Evaluation: How We Test Krisp\u2019s Technology","datePublished":"2022-09-23T12:47:00+00:00","dateModified":"2023-07-23T09:10:58+00:00","mainEntityOfPage":{"@id":"https:\/\/krisp.ai\/blog\/noise-cancellation-quality-evaluation\/"},"wordCount":1836,"commentCount":4,"publisher":{"@id":"https:\/\/krisp.ai\/blog\/#organization"},"image":{"@id":"https:\/\/krisp.ai\/blog\/noise-cancellation-quality-evaluation\/#primaryimage"},"thumbnailUrl":"https:\/\/krisp.ai\/blog\/wp-content\/uploads\/2022\/09\/Visulas-52.png","articleSection":["Engineering Blog"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/krisp.ai\/blog\/noise-cancellation-quality-evaluation\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/krisp.ai\/blog\/noise-cancellation-quality-evaluation\/","url":"https:\/\/krisp.ai\/blog\/noise-cancellation-quality-evaluation\/","name":"Noise Cancellation Quality Evaluation: How We Test Krisp\u2019s Technology","isPartOf":{"@id":"https:\/\/krisp.ai\/blog\/#website"},"primaryImageOfPage":{"@id":"https:\/\/krisp.ai\/blog\/noise-cancellation-quality-evaluation\/#primaryimage"},"image":{"@id":"https:\/\/krisp.ai\/blog\/noise-cancellation-quality-evaluation\/#primaryimage"},"thumbnailUrl":"https:\/\/krisp.ai\/blog\/wp-content\/uploads\/2022\/09\/Visulas-52.png","datePublished":"2022-09-23T12:47:00+00:00","dateModified":"2023-07-23T09:10:58+00:00","description":"Peek inside the Krisp Research Team\u2019s noise cancellation quality evaluation. We\u2019ll show you how we conduct speech quality testing on our NC algorithm.","breadcrumb":{"@id":"https:\/\/krisp.ai\/blog\/noise-cancellation-quality-evaluation\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/krisp.ai\/blog\/noise-cancellation-quality-evaluation\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/krisp.ai\/blog\/noise-cancellation-quality-evaluation\/#primaryimage","url":"https:\/\/krisp.ai\/blog\/wp-content\/uploads\/2022\/09\/Visulas-52.png","contentUrl":"https:\/\/krisp.ai\/blog\/wp-content\/uploads\/2022\/09\/Visulas-52.png","width":1000,"height":700,"caption":"noise cancellation quality evaluation"},{"@type":"BreadcrumbList","@id":"https:\/\/krisp.ai\/blog\/noise-cancellation-quality-evaluation\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/krisp.ai\/blog\/"},{"@type":"ListItem","position":2,"name":"Noise Cancellation Quality Evaluation: How We Test Krisp\u2019s Technology"}]},{"@type":"WebSite","@id":"https:\/\/krisp.ai\/blog\/#website","url":"https:\/\/krisp.ai\/blog\/","name":"Krisp","description":"Blog","publisher":{"@id":"https:\/\/krisp.ai\/blog\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/krisp.ai\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/krisp.ai\/blog\/#organization","name":"Krisp","url":"https:\/\/krisp.ai\/blog\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/krisp.ai\/blog\/#\/schema\/logo\/image\/","url":"https:\/\/krisp.ai\/blog\/wp-content\/uploads\/2024\/10\/K.png","contentUrl":"https:\/\/krisp.ai\/blog\/wp-content\/uploads\/2024\/10\/K.png","width":696,"height":696,"caption":"Krisp"},"image":{"@id":"https:\/\/krisp.ai\/blog\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/krispHQ\/","https:\/\/x.com\/krispHQ","https:\/\/www.linkedin.com\/company\/krisphq\/","https:\/\/www.youtube.com\/channel\/UCAMZinJdR9P33fZUNpuxXtg"]},{"@type":"Person","@id":"https:\/\/krisp.ai\/blog\/#\/schema\/person\/172d23b73915155e0ab4e97868216bd1","name":"Krisp Research Team","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/krisp.ai\/blog\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/49fc839d54b3ccba70e28ccaad1472a7?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/49fc839d54b3ccba70e28ccaad1472a7?s=96&d=mm&r=g","caption":"Krisp Research Team"},"url":"https:\/\/krisp.ai\/blog\/author\/research-team\/"}]}},"_links":{"self":[{"href":"https:\/\/krisp.ai\/blog\/wp-json\/wp\/v2\/posts\/9654"}],"collection":[{"href":"https:\/\/krisp.ai\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/krisp.ai\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/krisp.ai\/blog\/wp-json\/wp\/v2\/users\/65"}],"replies":[{"embeddable":true,"href":"https:\/\/krisp.ai\/blog\/wp-json\/wp\/v2\/comments?post=9654"}],"version-history":[{"count":12,"href":"https:\/\/krisp.ai\/blog\/wp-json\/wp\/v2\/posts\/9654\/revisions"}],"predecessor-version":[{"id":10139,"href":"https:\/\/krisp.ai\/blog\/wp-json\/wp\/v2\/posts\/9654\/revisions\/10139"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/krisp.ai\/blog\/wp-json\/wp\/v2\/media\/9661"}],"wp:attachment":[{"href":"https:\/\/krisp.ai\/blog\/wp-json\/wp\/v2\/media?parent=9654"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/krisp.ai\/blog\/wp-json\/wp\/v2\/categories?post=9654"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/krisp.ai\/blog\/wp-json\/wp\/v2\/tags?post=9654"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}