多模异构生物特征深度融合、关联推理及其一致性语义挖掘关键技术研究

It is a non-trivial task to excavate the underlying relations within multimodal biometrics due to the heterogeneity, complex distribution and non-static appearances, which also pose a rigorous challenge to the current data processing methods. To tackle these problems, this project intends to exploit a series of effective heterogeneous perceptual information processing algorithms via deep feature fusion, correlation inference and semantic consistency mining. The main contents are elaborated as follows: 1) Within the heterogeneous feature fusion, we shall mainly concentrate on studying the co-training associated with discriminative kernel sparse representation and multimodal convolutional deep belief networks, to establish the hierarchical computation theory and construct the joint appearance model; 2) Within the correlation inference, we shall primarily focus on employing the sparsity preserving item and Fisher discrimination term to exploit the feature correlations, whereby the inherent disciplines about the max-relativity between the heterogeneous features would be revealed; 3) Within the semantic consistency mining, we intend to adopt the sparse topic model and accumulated reconstruction error space to exploit the semantic relationships of multimodal biometrics and their collaborative learning scheme, thereby the semantic consistency model can be established. Remarkably, the aforementioned three aspects are closely related, and it is expected that a series of novel algorithms related to the adaptive and robust heterogeneous perception information processing would be prospectively addressed for robust multimodal biometric recognition.

多模生物特征的异构性、分布复杂多样性和非静态性加大了该领域潜在关系的抽取难度，并对现有数据处理方法提出了严峻的挑战。本项目拟通过特征深度融合、关联推理及其一致性语义挖掘三个方面推进多模生物特征识别中异构感知信息的有效处理：1）在异构特征融合方面，拟从合作学习与判别性核稀疏表达以及多模卷积深度置信网络来建立符合多模特征理解的层次化计算理论与联合表观模型；2）在关联推理方面，拟同时引入稀疏保持项和判别性保持项进行特征关联挖掘，从而揭示异构特征相关性最大化的内在规律；3）在语义一致性描述方面，分别从稀疏主题建模和累积重构代价向量空间进行多模生物特征的语义关联建模及其协同机制学习，并构建其语义一致性表达模型。以上三个方面紧密关联，预期提出一批适用面广和智能性强的异构感知信息理解新算法。

项目利用计算机视觉和模式识别的创新理论对 “多模异构生物特征深度融合、关联推理及其一致性语义挖掘”这一课题的深入研究；同时，根据人脸和语音两种异构模态各自特性，在模型分析、算法实验和应用研究三个层面探讨并研究融合这两种模态的基本理论和方法，提出并验证了若干基于人类视觉识别和机器学习识别的融合算法用于提升身份鉴定系统的各方面性能。通过本项目的研究，目前取得的主要成果有以下几点：1）创新的提出了一种基于深度学习的人脸和语音双模态的深度融合方法；2）提出了一种基于人脸和语音的时序建模和深度融合方法；3）提出了一种基于注意力模型的人脸和语音深度融合的说话人标注方法；4）提出了基于一致性自编码器的异构媒体跨模态匹配计算方法；5）提出了一种基于判别性张量的人脸修复方法；6）提出了一种多尺度小波变换的图像清晰化方法；7）一种结合粒子滤波和增量学习理论的唇动区域追踪方法；8）基于时序邻近词袋模型的运动特征表示学习；9）基于中心距离和角度融合的联合特征表示学习方法。本年度已发表j基金标注SCI论文（5篇），国内核心期刊2篇，国际EI会议论文（6篇)，培养硕士生6名（1名已毕业）.