一种具有迁移学习能力的人机语音情感交互研究

From the point of view of artificial emotion, this item makes a systematic study on the speech emotion interaction in Human-Computer Interaction based on emotion perception oriented to emotional speech. First of all, focuses on the problem of data incompleteness in target domain, we introduce transfer learning, and propose the emotion information transfer strategy with instance transfer and feature transfer. By means of the cumulative learning based on the existing knowledge, we can recognize the speech emotion state in the new application. This approach can solve effectively the problem of model inadequate training with a little train data in different applications and improve system performance for higher emotion recognition rate. Then, we adopt the hierarchical decision network with affection feedback, and design the predicative analysis and evaluation model by tracking the affective feedback to adjust affective decision-making. At last, the item realizes the speech emotion interaction system in HCI with affective feedback. This item’s research can advance the theoretical research of affective computing. It provides a new train of thought and reference for emotional feature representation and recognition of little speech data in target domain. This study will contribute to broaden the application of speech emotion recognition in artificial intelligence fields and explore the importance of affective cognitive in decision-making.

本项目主要面向自然语音情感数据，从人工情感的角度出发，系统地研究基于情感认知的人机语音情感交互方法。首先，针对目标领域语音情感数据样本有限或缺失条件下的情感分析应用问题，引入迁移学习机制，提出基于样例和特征变换的语音情感信息迁移策略，实现基于已有知识的可积累性学习辅助完成新应用环境下的语音情感识别任务，有效解决不同应用环境下，待分析样本信息缺少而导致的模型训练不足问题，提高系统的整体识别性能。然后，采用具有情感反馈的分层决策网络算法，通过跟踪交互者反馈的情感波动信息，设计预测分析与评价模型，通过自知式的学习方式调整情感决策，最终实现具有情感反馈的人机语音情感交互系统。本项目的开展既能促进情感计算的基础理论研究，为在数据样本较少条件下的语音情感特征的表达、分析与识别提供新的思路和参考，又能够拓展其在人工智能领域的应用，探索情感认知在决策方面的重要性。

在国家自然科学基金青年基金项目：“一种具有迁移学习能力的人机语音情感交互研究”资助下，大连大学董婧研究组在项目执行期间，结合人体姿态识别及图像增强技术，利用卷积神经网络及长短时记忆网络设计并实现了自然语音的情感识别系统，取得了初步的研究成果。针对特定应用场景的实时性语音情感识别问题，提出了基于一维卷积神经网络的端到端语音情感识别算法，并在此基础上采用自适应阈值的Gamma变换进行语谱特征增强，构建基于二维卷积神经网络和特征融合的情感识别系统。同时，结合人体姿态识别技术尝试开展多模态的情感识别研究。项目执行期间，研究组发表论文11篇，录用1篇，其中，EI检索6篇，培养研究生4人，国际合作交流正在继续开展中。