An Iterative Mask Estimation Approach to Deep Learning Based Multi-Channel Speech Recognition

Publication date: Available online 26 November 2018Source: Speech CommunicationAuthor(s): Yan-Hui Tu, Jun Du, Lei Sun, Feng Ma, Hai-Kun Wang, Jing-Dong Chen, Chin-Hui LeeAbstractWe propose a novel iterative mask estimation (IME) framework to improve the state-of-the-art complex Gaussian mixture model (CGMM)-based beamforming approach in an iterative manner by leveraging upon the complementary information obtained from different deep models. Although CGMM has been recently demonstrated to be quite effective for multi-channel, automation speech recognition (ASR) in operational scenarios, the corresponding mask estimation, however, is not always accurate in adverse environments due to the lack of prior or context information. To address this problem, in this study, a neural-network-based ideal ratio mask estimator learned from a multi-condition data set is first adopted to incorporate prior information, obtained from the speech/noise interactions and the long acoustic context, into CGMM-based beamformed speech that has a higher signal-to-noise ratio (SNR) than the original noisy speech signal. Next, to further utilize the rich context information in deep acoustic and language models, voice activity detection information, obtained from speech recognition results, is then used to refine mask estimation, yielding a significant reduction in insertion errors. During testing on the recently launched CHiME-4 Challenge ASR task of recognizing 6-channel microphone array speech, the propo...
Source: Speech Communication - Category: Speech-Language Pathology Source Type: research