Review on automatic speech recognition quality estimation (ASR QE).The application of ASR QE in ASR system combination for both single-microphone multiple-ASR system task and on multiple-microphone multiple-ASR system task.Ranking the system combination inputs based on predicted quality.Management of tied ranks.Automatically finding the optimum level of combination for each segment. Recognizer Output Voting Error Reduction (ROVER) has been widely used for system combination in automatic speech recognition (ASR). In order to select the most appropriate words to insert at each position in the output transcriptions, some ROVER extensions rely on critical information such as confidence scores and other ASR decoder features. This information, which is not always available, highly depends on the decoding process and sometimes tends to overestimate the real quality of the recognized words. In this paper we propose a novel variant of ROVER that takes advantage of ASR quality estimation (QE) for ranking the transcriptions at segment level instead of: i) relying on confidence scores, or ii) feeding ROVER with randomly ordered hypotheses. We first introduce an effective set of features to compensate for the absence of ASR decoder information. Then, we apply QE techniques to perform accurate hypothesis ranking at segment-level before starting the fusion process. The evaluation is carried out on two different tasks, in which we respectively combine hypotheses coming from independent ASR systems and multi-microphone recordings. In both tasks, it is assumed that the ASR decoder information is not available. The proposed approach significantly outperforms standard ROVER and it is competitive with two strong oracles that exploit prior knowledge about the real quality of the hypotheses to be combined. Compared to standard ROVER, the absolute WER improvements in the two evaluation scenarios range from 0.5 to 7.3 .
@cite_57: Minimum variance beamformers are usually complemented with diagonal loading techniques in order to provide robust ness against finite sample size effects. In spite of its usefulness and popularity, some of the insights of this interesting technique have remained inconspicuous for decades. This work presents a new asymptotic performance study that will shed some light on the finite sample size behavior of this spatial filter. The asymptotic description establishes the existence of an optimum loading factor, which can be estimated from the received data.
@cite_48: In this work, we present our progress in multi-source far field automatic speech-to-text transcription for lecture speech. In particular, we show how the best of several far field channels can be selected based on a signal-to-noise ratio criterion, and how the signals from multiple channels can be combined at either the waveform level using blind channel combination or at the hypothesis level using confusion network techniques to improve the accuracy of a far field lecture transcription system. Using the techniques described here, we ran a series of experiments on the test set used by the US National Institute of Standards and Technologies for the RT-05S evaluation. For the multiple distant microphones (MDM) task of RT-05S, our system achieved a word error rate of 38.5 which represents an improvement of over 13 absolute compared to the best reported results in the RT-05S evaluation. Index Terms: far-distance, automatic speech recognition
@cite_4: Speech recognition in a realistic noisy environment using multiple microphones is the focal point of the third CHiME challenge. Over the baseline ASR system provided for this challenge, we apply state of the art algorithms for boosting acoustic model learning and hypothesis rescoring to improve the final output. To this aim, we first use the automatic transcription of each channel to re-train the acoustic model for that channel and then we apply linear language model rescoring to find a better solution in the n-best list. LM rescoring is performed using an efficient set of N-gram and Recurrent Neural Network LM (RNNLM) trained on a wisely-selected text set. In the experiments, we show that the proposed approach improves not only the individual channel transcription, but also the enhanced channels produced by MVDR and delay-and-sum beamforming.
In addition to DS, MVDR and other signal-based combination approaches @cite_57 , the automatic selection of the best channel can be used to perform signal enhancement. The simplest method to do this consists in measuring the signal-to-noise ratio (SNR) of the recorded signals @cite_48 , assuming that the channel with the highest SNR will be the easiest to transcribe. Choosing the channel for which the ASR engine has generated the highest confidence score can also represent a viable solution. For instance, in a previous work with data from the aforementioned CHiME-3 challenge, we successfully applied channel selection based on sentence confidence estimation @cite_4 to select the best microphone.