面向图像标注的上下文感知深度核映射网络方法研究

Automatic image annotation, one of most important components for real-time Internet content surveillance and healthy Internet contents, is a popular yet challenging research topic in image processing and pattern recognition. The current methods based on deep learning for image annotation have obtained good performance, but there is no available complete theory to explain the underlying principle of deep network. Although the theory of support vector machines based on kernel functions is complete, especially multiple kernel learning can further improve the performance of support vector machines, they are not suitable for large-scale image annotation datasets. At the same time, the traditional annotation methods ignore the context information of the images. In this project, we will investigate image annotation task by combining kernel methods, context-aware methods and deep learning together and conduct the following works: (1) By analyzing the weakness of deep kernel networks, we build an equivalent end-to-end deep kernel map networks and learn deep kernel map features, which can reduce the complexity of deep kernel computation in the deep kernel networks, and it is suitable for large-scale image annotation datasets; (2) By integrating the prior context information of the image into deep kernel map network, we propose the context-aware deep kernel map networks, which can learn more powerful context-aware deep features for image annotation and improve the annotation performance; (3) By considering that the number of labeled images is much less than the number of unlabeled images in the real annotation tasks, we propose semi-supervised learning algorithm for context-ware deep kernel map networks to make full use of all the available information and then to improve the annotation performance. The scientific significance of this project includes: (1) the approximation method of map function in Hilbert space for deep kernels through network optimization; (2) the framework of context-aware deep kernel map networks and its associating supervised and semi-supervised algorithms which integrate the context information in the image and between the images to improve the annotation performance. Investigation of this project is expected to publish several high quality original research papers in international journals and conferences and also application for at least one patent. The goal of this project is to enrich the theory of image annotation and kernel learning, to extend the applications of deep learning and to significantly improve the performance of image annotation for the healthy development of Internet contents.

图像自动标注是保障网络图像实时监控和内容健康的重要技术手段，是图像处理和模式识别领域的研究热点和难点。当前基于深度学习的图像标注方法取得了不错的效果，但尚未有完备理论阐释学习过程，而基于核方法的支持向量机有完备理论，但不适合大规模图像集。本项目拟借助深度学习思想，结合多核学习，面向图像标注问题，进行以下研究：（1）针对深度核学习网络的缺点，构建端对端的深度核映射网络，学习深度核映射特征，适用于大规模图像标注集；（2）融合图像不同区域的上下文先验知识，构建上下文感知深度核映射网络，学习识别力更强的深度特征，用于图像标注；（3）针对标注图像的数量远少于未标注图像，构建半监督上下文感知深度核映射网络，提高图像标注性能。关键科学意义在于：（1）最优化估计深度核函数在希尔伯特空间的核映射函数；（2）上下文感知深度核映射网络监督和半监督地学习图像的上下文关系。本项目利用深度学习推动核学习在大规模图像标注任务的性能，发展多核学习理论，扩大深度学习的应用场景。研究期间预期发表8篇学术论文，3-4篇发表至知名国际刊物，且主要成果生成发明专利；通过本项目研究，显著提升图像标注性能，为网络信息健康发展服务。

图像自动标注是图像处理和模式识别领域的研究热点，当前基于深度学习的图像标注方法取得了不错效果，但面临两个难点：其一是深度学习缺乏完备理论解释其学习过程；其二是深度学习需要大规模强监督训练图像抵抗过拟合。本项目针对以上两点不足，提出基于多核学习的深度核映射网络学习方法，凭借核学习的完备表示理论，解释网络结构选择，线性复杂度使网络适用于大规模图像集，同时在网络中自适应学习图像结构关系，提升图像标注的性能。. 本项目面向图像标注及其图像分类问题，结合深度神经网络和多核学习，主要进行了三项研究内容：（1）针对深度多核学习网络复杂度较高的缺点，借助SVD矩阵分解，学习深度非线性核函数在希尔伯特空间的近似高维深度核特征，将深度多核学习网络近似为端对端的深度核映射网络，大大地降低了深度核函数的计算复杂度，使其适用于大规模图像标注集；（2）在深度多核函数学习过程中，将图像内不同区域的上下文先验知识融入网络，构建上下文感知深度核映射网络，自适应学习图像内不同区域的关系，学习表达能力更强的深度特征，提升图像标注性能；（3）针对标注图像的数量远少于未标注图像，提出半监督上下文感知深度核映射网络，将图像间结构信息融入深度核特征学习，进一步提升图像标注性能。以上三项研究内容及相关算法均在图像标注标准数据集ImageClef、Corel5K，NUSWIDE等进行测试，均达到国际先进的性能。. 本项目目前共发表国际期刊论文6篇，国际会议论文4篇，其中包含了模式识别顶级期刊Pattern Recognition和主流期刊Neurocomputing等，提交了两项发明专利，目前处于实质审查阶段。通过本项目研究，发展了核学习理论，提出了新型的深度核映射网络结构，并成功地应用到图像标注及图像分类任务，提升了相关性能，为互联网图像内容自动检测和标注提供了新的算法。