基于内在与潜在语义特征的声音段落级语义识别方法研究

The semantic recognition of a sound (non-speech) is one of the core contents of the research on perceiving and understanding a sound. However, most researches on computational auditory scene analysis and acoustic event detection only focus on the recognition of local acoustic objects in the low-level semantic, and rarely explore on the recognition of a whole sound segment, which refers to a sound with the given duration in this project, in a global semantic level. As a novel research, the segment-level semantic recognition of a sound has many difficult problems which require to be solved. In this project, the segment-level semantic recognition is carried out based on the extraction of the internal and latent semantic features. The internal semantic can be directly obtained from the context of a sound segment, and the latent semantic can be obtained based on extracting the abstract information from several similar sound segments labled with human experience and knowledge. The main work in this project includes constructing and optimizing a suitable codebook for representing semantic feature well, extracting the internal and latent semantic features respectively, providing more available prior knowledge based on the background information of a sound segment, and recognizing the whole semantic from a sound segment by comprehensively utilizing the above two types of features and the prior knowledge. This research not only has important theoretical significance but also has practical value in improving cognitive ability of sound for computers and in promoting their real applications.

非语音声音的语义识别是声音感知与理解研究的核心内容之一。然而目前这方面的研究工作，无论是声学事件检测还是计算场景分析大多只关注于声音中局部声学对象的低层语义识别问题，缺乏对声音段落（本项目中指一定时长的声音）整体语义的识别研究。声音段落级语义识别是一个新兴的研究方向，还有许多问题亟待解决。本项目基于声音的内在和潜在语义特征来识别声音段落的整体语义。其中，内在语义是指可以直接依据声音段落本身内容来获得的语义，潜在语义是指必须借助人类经验知识从若干相近声音段落中凝练出的抽象语义。项目的主要研究内容包括：适合声音段落语义特征表示与提取的码本构建与优化、声音段落的内在与潜在语义特征提取、能提供更多语义识别先验知识的声音背景信息提取，以及结合上述两类语义特征和先验知识的声音段落整体语义识别。本项目的研究工作对提高计算机声音的认知能力，进而推动其走向现实应用具有重要的理论意义和实用价值。

本项目重点开展基于声音的内在和潜在语义特征来识别声音段落整体语义的研究。在项目的执行过程中严格按照项目计划书要求开展工作，已完成了所有的研究计划内容，并对部分研究内容进行了拓展性研究。在如下几方面取得了重要研究进展：① 基于L1正则稀疏约束及迹比大间隔子空间学习的表示性好的声音码本构建方法，以及基于Fisher判别准则字典学习的可区分性强的声音码本构建方法；② 基于非凸稀疏约束与TV正则，以及交叠分组稀疏约束的适合声音段落语义识别的码本优化方法；③ 基于BoAW框架的适合声音段落语义提取的非线性时序特征学习方法；④ 基于周期分解及张量分解的声音特征向量序列的内在语义特征提取方法；⑤ 基于偏最小二乘法以及概率与核偏最小二乘法的声音特征向量序列的潜在语义特征提取方法；⑥ 基于最小统计量追踪及听觉摘要统计量的声音段落背景信息的提取方法。. 目前项目组已经按计划任务书的要求完成了全部研究内容，提出了若干具有自主知识产权的语义识别方法。在本领域高水平期刊和会议上发表论文12篇，其中SCI收录的论文8篇，论文被他引29次，其中SCI他引18次。12篇论文全部是EI检索源。同时，由科学出版社出版了国内首部系统讨论声学事件检测理论与方法的专著。已获得发明专利授权3项。培养博士研究生3名，硕士研究生14名。此外，1本语音信号处理的教材即将于2019年3月由清华大学出版社出版。