{"id":9563,"date":"2022-08-29T23:11:45","date_gmt":"2022-08-29T19:11:45","guid":{"rendered":"https:\/\/krisp.ai\/blog\/?p=9563"},"modified":"2025-03-11T18:34:08","modified_gmt":"2025-03-11T14:34:08","slug":"speech-quality-measurement","status":"publish","type":"post","link":"https:\/\/krisp.ai\/blog\/speech-quality-measurement\/","title":{"rendered":"Speech Quality Measurement Algorithms and Testing Technology"},"content":{"rendered":"

Estimating the quality of speech is a central part of quality assurance in any system for generating or transforming speech.\u00a0<\/span><\/p>\n

Such systems include telecommunication networks or speech processing or generating software. The speech signal in their output suffers from various degradations inherent to the particular system.<\/p>\n

With background noise cancellation, an algorithm could leave remnants of noise in an audio snippet or partially suppress speech (see Fig. 1). Or, a telecommunication system could also suffer from packet loss and delays. Meanwhile, audio codecs can introduce unwanted coloration just like speech-to-text systems deliver unnatural sounds.\u00a0<\/span><\/p>\n

A more straightforward approach to speech quality testing is conducting listening sessions, resulting in subjective quality results.\u00a0<\/span><\/p>\n

A group of people listens to recordings under controlled settings. They\u2019re then asked to provide normalized feedback (e.g. on a scale from 1 to 5). Responses are then aggregated into a single quality value, called Mean Opinion Score (MOS).\u00a0<\/span><\/p>\n

Aggregation is necessary to avoid subject bias. There are standard guidelines for conducting such listening tests and for further statistical analysis of the results that yield the MOS values (see ITU-T P.830, ITU-R BS.1116, ITU-T P.835).<\/span><\/p>\n

\"degraded<\/p>\n

Fig. 1 An example of a degraded speech signal and its reference<\/span><\/p>\n

In some circumstances, conducting listening sessions for collecting MOS is infeasible, laborious, or costly due to the large volume of the data to be tested.\u00a0<\/span><\/p>\n

This is where <\/span>automated speech perceptual quality measures<\/span><\/i> come into play. The aim is to replicate the way humans evaluate speech quality, avoiding subject bias inherent with subjective listening panels.<\/span><\/p>\n

In short, the goal is to objectively approximate or predict voice quality in the presence of noise. The perceptual speech quality estimation is assessed in two cases, depending on whether clean reference speech can be compared with the system output or not (depicted in Fig. 2).<\/span><\/p>\n

\"<\/p>\n

Fig. 2 Schematic view of intrusive and non-intrusive measures (source: ITU-T P.563)<\/span><\/p>\n

Double-ended (intrusive) speech quality assessment measures<\/span><\/h2>\n

The model, in this case, has access to both the reference and output audio of the speech processing system. Its score is given based on the differences between the two. Please note that model and measure are used interchangeably in this article.\u00a0<\/span><\/p>\n

To mimic a human listener, automated methods should \u201ccatch\u201d speech impairments that are detectable by the human ear\/brain and assign them a quantitative value. It\u2019s not enough to compute only mathematical differences between the audio samples (waveforms).\u00a0<\/span><\/p>\n

Such algorithms need to model the human psychoacoustic system. In particular, they\u2019re expected to capture the ear functionality as well as certain cognitive effects.\u00a0<\/span><\/p>\n

To mimic human hearing capability, the model obtains an \u201cinternal\u201d representation of the signals that mimics the transformations happening within the human ear. Signals of different frequencies are detected by separate areas of the cochlea (see Fig. 3). This \u201ccoverage\u201d of frequencies isn\u2019t linear. <\/span>As a consequence, the audible spectrum can be partitioned into frequency bands of various widths<\/em> the ear perceives as equal<\/em> widths.<\/span><\/p>\n

This phenomenon is modeled by psychoacoustic scales such as Bark and Mel scales (e.g. PESQ, NISQA double-ended, and PEAQ basic version), or by filter banks like the Gammatone filter bank (e.g. PEMO-Q, ViSQOL, PEAQ advanced version).<\/span><\/p>\n

\"auditory<\/p>\n

Fig. 3 Schematic view of auditory system (<\/span>source<\/span><\/a>)<\/span><\/p>\n

Other voice quality aspects to consider are:<\/span><\/p>\n