面向远场并发声学事件的深度实时分离研究

It is almost impossible to predict the spatial locations of acoustic events. The microphones recording the acoustic signal are often far away from the acoustic events, enabling the direct-path signal to be very weak. Especially, the long-term reverberation is unknown for most scenarios, and therefore, the acoustic signals of multiple acoustic events are deeply coupled with each other. It is a challenging issue to separate the reverberated mixture of the acoustic signal of multiple events. In the past several decades, some unsupervised methods have been presented to separate the acoustic mixture of special sound source signals with some given assumptions. However, those methods are generally designed for some special events with unacceptable latency and high computation complexity, which are difficult to satisfy the requirements. The real-time applications requires the separation to be realized with low latency and high computational efficiency。Neither the microphone array technique nor the deep learning is capable of well treating the mixture of acoustic events. This research combines the techniques of array and deep learning，which is described as follows: 1. Real-time multiple source localization, which provides the clue for separation; 2. Time-frequency bin classification based on the spatial information of acoustic events, which separate the direct-path signals of events; 3. Reverberation reduction based on generative adversarial network, which reduces the reverberation that is remained in the separated signals. This research eventually summarizes these three researches to construct an intelligent system of acoustic event perception.

声学事件发生的空间位置具有不可预知性，麦克风通常远离事件而形成一个远场，而且远场下的长时混响未知，使得多个并发事件的声信号形成深度耦合。远场并发声源分离是一个深度解耦合的挑战性难题。前人利用非监督方法能够在某些条件下实现一定程度的分离。然而，这些方法针对特定声信号，时间延迟较大，且计算复杂度较高，难以满足实时要求。声学事件的实时分离要求各环节的算法同时具备低延迟、高计算效率等要求，单纯依靠麦克风阵列或者深度学习，难以解决问题。本项目试图将二者有机结合，发展声学事件实时分离的理论和方法，具体从以下方面展开研究：1.多声源实时定位，在短时片段上实现定位，降低延迟，为分离提供线索；2. 基于空域信息的有监督掩模聚类，由掩模确定各声源的导向矢量，从而实现直达声信号的分离；3. 基于深度对抗的混响抑制方法，去除残留于直达声信号中的混响。项目整合成果，形成智能化的声学事件感知系统。

麦克风阵列由2个以及2个以上的声学传感器，按一种特定的拓扑结构组成，具有一定的空间滤波能力。麦克风阵列将人们从“麦克风近讲”模式的束缚中解放出来，实现更加自然的远场人机交互，通过相应的波束形成算法，达到语音增强，分离的效果。通过分析多通道信号，能够实现估计声源数目，发现声源方位，波束自适应跟随声源移动的功能，进而达到环境声学感知的功能。前人对声学感知进行了卓有成效的研究，但在两个方面受到瓶颈性的局限。其一，单通道的前端处理方法丢失声源空域信息，且没有长时依赖性。其二，基于阵列的方法没有给出去混响的解决方法。因此，需要全面考虑针对声学场景的前端阵列处理方法。本课题对这些问题进行了研究，设计出多种麦克风阵列电路板，且仿真和模拟生成上百小时的数据库，建立了一套实时定位与分离系统。基于该系统，本课题研究了单通道的语音分离和增强，利用单通道到多通道的矩阵转换算法，实现模型的长时依赖能力，提高语音分离后的人的感知效果。本课题针对多说话人的场景利用声纹识别技术增强模型的环境感知能力，实现分离识别一体化功能，提高系统的实用性。针对多通道的场景，充分利用麦克风的空间信息，本课题还结合传统的差分麦克风阵列原理，构建了基于差分计算的深度神经网络 (DMANet)， 不仅语音分离和增强效果达到当时最优，模型小型化得到更进一步优化提升。语音发自人的声道，对人的发音机理研究可以提供更好的语音信号特征，所以本课题还从嗓音学入手，分析人在发元音基频，提出一种RS-CEPS方法，计算更加准确的频谱包络特征，并设计声纹识别模型进行个性特征验证。