基于深度学习的细粒度图像分类识别方法研究

Fine-grained visual categorization (FGVC) is a very difficult task due to the high degree of appearance similarity between inter-class objects and the intra-class large pose variations. Therefore, it is reasonable to first locate the object parts and then extract features from each part for recognition. Nevertheless, whether it is necessary to explore the relationships between object parts to improve the performance, or how to improve the performance between object classes with high degree of appearance similarity but large semantic distance are still not fully studied. Our preliminary experiments indicate that embedding relevant prior knowledge in the designing of models has a certain degree of feasibility, however, more experiments are needed to validate the performance of our models. To this end, this project will systematically study the FGVC problem based on the utilization level of the relationships between object parts in the designing of the models. The project first unites the part detection and feature learning in a deep learning model for classification based on the independent prediction of each object part. Then it models the dependency between object parts based on the hypothesis that our human vision internally automatically explores the relationships between object parts for fine-grained recognition, and constructs the part and image features for classification. Finally, it treats all parts as an integrity, and explores a multi-task learning model based on the embedding of poses and category-grained structures to improve the performance between object classes with high degree of appearance similarity but large semantic distance. The project will advance and enrich the methods and theories of the FGVC, possessing a certain degree of value of the research.

细粒度图像分类识别由于目标间表观极为相似且同类别目标姿态变化较大而难度较高。其识别过程往往是先定位部件而后由此构造特征进行分类识别，但其是否需挖掘部件间结构关系，以及如何提升表观极为相似但语义距离较远类别间的性能，仍尚未展开深入研究。申请人前期实验表明在模型设计过程中嵌入相关先验知识具有一定可行性，但仍需进一步实验以验证性能。基于此，本项目将依据模型设计过程中部件相关性利用程度展开系统研究。项目首先探索各部件独立预测情况下，将部件检测与特征学习耦合至同一深度学习模型进行分类识别；其次基于人类视觉精细识别内在挖掘部件间结构关系这一假设，建模部件间相互依存关系，并由此构造部件与图像特征进行分类识别；最后将各部件合为整体，开发以姿态与类别粒度结构嵌入的多任务学习模型，提升表观极为相似但语义距离较远类别上的分类识别性能。项目研究将一定程度发展与丰富细粒度图像分类识别方法与理论，具有一定的科研价值。

项目主要从模型结构创新和判别特征学习两方面对细粒度图像分类识别展开了系统研究。在模型结构方面，项目开发了组稀疏深度堆叠网络模型，模型通过挖掘深度神经网络中神经元的组结构关系，提升了细粒度图像分类识别精度；项目开发了跨类别跨语义跨尺度的细粒度图像分类识别深度卷积网络模型，模型充分利用细粒度目标物体结构相似性，通过开发相应的相互协作机制提升了模型性能。在特征学习方面，项目在深度卷积网络中开发了基于语义增强的特征学习方法，方法在不增加特征维度前提下使子特征具有语义可解释性和较强的判别性，从而提升特征整体判别性能；项目开发了基于类语义距离约束的特征学习方法，方法采用类元信息构建层级结构树正则化模型学习；项目进一步开发了针对大类别情况下的特征学习方法，方法通过学习细粒度图像共性特征提升特征的判别性。..项目研究获得以下重要结果：..1. 从图像局部语义区域提取判别特征，对于细粒度分类识别是一种切实可行的策略。.2. 细粒度目标物体结构相似性可以被充分利用以约束局部语义区域定位的精度和效率。.3. 不同尺度特征必须有机耦合到一起方能有效提升细粒度特征判别力，反之会降低其判别性能。.4. 图像子特征可与局部语义区域对应，一方面可提升图像整体特征的细粒度判别力，另一方面可使其具有可解释性。.5. 细粒度类别间语义距离可有效引导判别特征学习，从而降低表观极为相似但语义距离较远类别间的错判率。.6. 在大类别情况下，通过匹配模型预测分布与均匀分布可有效提升模型性能。.7. 采用网络扩充数据训练细粒度分类识别模型时，若扩充数据中包含少量噪声图片，可适当提升模型性能。..综合而言，项目研究了细粒度图像分类识别机制并对其进行了理论建模，构建了具有可解释性的细粒度判别特征，挖掘了细粒度共性特征学习机制，开发了可适用于大类别的快速分类识别方法。项目研究在一定程度上丰富和发展了细粒度图像分类识别理论和方法。