基于深度神经网络的噪声鲁棒性语音识别方法研究

Improving the noise robustness of automatic speech recogntion (ASR) system in real scenarios is one of the key challenges and hot topics for the application of speech recognition. But as the statistical properties of both speech and noise signals are extremely complicated, and many assumptions are made for convenient theorectical derivation in traditional noise-robust methods, the improvement of recognition performance is limited to some extent, and the advantages of different methods can not be combined properly. With the successful application of deep neural network (DNN) for the acoustic modeling of large vocabulary continuous speech recognition (LVCSR), the research on DNN for noise robustness is expected to make up the defects of traditional noise-robust methods and bring the breakthrough. This project aims to fully exploiting powerful capability of DNN for nonlinear modeling. On the one hand, DNN is used in front-end for feature extraction, e.g., to learn the mapping function between the noisy speech and "clean speech". On the other hand, DNN is used for acoustic modeling in back-end, e.g., to combine different front-end algorithms by using Hierarchical DNN. Besides, two DNNs of both front-end and back-end can also be concatenated for joint optimization. Hopefully it can further improve the recognition performance in noisy environments. Meanwhile, part of research results for this project is also of great importance for the basic problems of signal processing area, e.g., speech enhancement.

提高语音识别系统在实际环境下的噪声鲁棒性是语音识别实用化的关键难点和研究热点之一。但由于语音和噪声信号的统计特性都极其复杂,而传统噪声鲁棒性方法为了方便理论推导又作了诸多假设,从而很大程度上限制了识别性能的提高,并且不同方法之间的互补优势也无法很好结合。随着深度神经网络(DNN)在大词汇量连续语音识别声学建模中的成功应用,DNN结合噪声鲁棒性问题的研究,将有望弥补传统噪声鲁棒性方法的缺陷,并带来突破性进展。本项目旨在充分利用DNN强大的非线性建模能力,一方面将DNN用于前端特征提取，比如学习带噪语音和"干净语音"之间的映射关系;另一方面将DNN用于后端声学建模,比如使用Hierarchical DNN将不同前端算法加以融合;此外前后端两个DNN还可以联合优化，以期最大程度的提高噪声环境下语音识别的性能。同时，本项目部分研究成果对语音增强等信号处理领域的基础问题也具有重要意义。

语音识别是实现智能人机交互的关键技术之一，但在实际复杂场景下，由于各种干扰带来识别率的大幅降低，用户体验并不好。语音识别的噪声鲁棒性研究始于上世纪 80 年代，但由于实际中的语音和噪声都是统计特性极其复杂的信号，使得这个问题始终没有得到很好的解决。本项目基于近年来深度神经网络（DNN）在语音识别领域的成功应用，从如下方面展开了基于DNN的噪声鲁棒性方法研究。首先，采用基于DNN的预处理方法对输入语音降噪，这种方法的优势是识别系统的特征提取和声学建模不需要做任何修改。其次，采用基于DNN的特征映射方法将带噪语音的声学特征映射到干净语音的声学特征，并且可以和后端声学建模的DNN进行联合训练，从而可以达到更好的识别效果，在Aurora4和SSC等标准数据库上取得了论文发表时的最佳结果。此外，我们还探讨了将同样的框架应用到特定人分离识别问题中，在实际数据上取得了很好的效果。此项目的成果可以不仅可以应用在噪声鲁棒性语音识别中，还可以扩展到很多语音相关的领域，比如语音检测，频带扩展等，有着非常广阔的应用前景。