基于深度复合语境的视频事件识别模型研究

The effective recognition, intelligent analysis and understanding of video data have become one of the most important development directions in information processing field. Although many researchers have proposed various methods for video event analysis and recognition, there still exist some limitations in them. Firstly, since the existing methods did not take full advantage of the mechanism of human visual perception, the global information and local relationships of events cannot be well explored in them. Secondly, most existing methods represent the video data heavily depend on the single and low-level feature descriptor. Thus, it is difficult for them to build a bridge to connect the low-level and the semantic-level features. In order to overcome these limitations, we first expand the feature representation space of video data based on the mechanism of human visual perception. As a result, the video events can be represented by fusing three types of features extracted from appearance, motion and context spaces. In our project, the video saliency mechanism is enriched and its model is established using Bayesian Network. Then, we regard the events as basic units, and employ the manifold learning, dictionary learning and global scene information to establish the global and deep representation model of video events. At the same time, the mid-level contextual deep model which reflects the local relationships among events is also constructed based on the time-space relationship and local scene information. At last, a composite deep model is established for video event recognition, which could improve the performance of event recognition. Moreover, the proposed model can also enrich the theory of video analysis.

对视频事件的高效识别与智能分析，已成为信息处理领域重要的研究方向，有重要的理论价值和应用前景。但目前的研究仍存在很多问题与挑战：没有充分利用人类视觉认知机理；缺乏对视频事件有效的全局表示，事件内部的局部关系描述不够深入；视频表示上多依赖于底层特征，多局限于单层特征抽取模型；多处理由简单语义动作或交互构成的事件。本项目首先从视觉认知机制出发，拓展视频的特征表示空间，在多个域提取描述事件的表征、运动及语境特征。其次，在视频事件识别中引入视觉注意机制，基于概率图模型构建视频事件注意模型。然后，建立不同层次视觉特征与事件高层语义的桥梁：以事件为基本单元，流形学习、字典学习为工具，融合全局场景，构建视频事件全局深度表示模型；并基于事件对象间的时空关系，融合局部场景，构建反映事件局部关系的中层语境深度模型。最后，构建基于深度复合的视频事件识别模型，拟提高视频事件识别模型的性能，丰富视频分析模型和理论。

本项目旨在深入挖掘高维海量视频数据所蕴含的特性和规律，探索人类视觉认知机制在视频分析领域的应用。通过融合多层次语境信息，构建深度复合视频事件识别模型，丰富视频分析相关的理论与方法，从而提高视频分析的效率和智能化水平。. 本项目的研究内容为基于深度复合语境的视频事件识别。项目在研期间，项目组提出并实现了：一系列用于视频事件识别的图像或视频帧的降维、表达及描述的模型与方法；一系列融合特征、语义和场景语境信息先验的模型与方法；并给出视频深度复合语境识别模型；进一步将提出的方法扩展和应用到其它机器学习经典领域，进一步验证了模型与方法的有效性与稳健性，取得了预期研究成果，达到了预计效果。.项目是以社会实际需求为背景，旨在对影响视频分析、识别效率的机理展开研究；融合认知、机器学习、视频处理等学科的理论与方法，从多角度展开研究，研究成果可以在一定程度上促进多视角视频数据分析和处理的相关研究，并为相关的研究领域提供理论和技术上的借鉴。.目前，项目组已经将本项目的部分研究内容及结果，如视频中人的行为分析、场景分析、目标检测等技术应用于智慧康养和智慧教育领域中，这将对解决我国康养和教育中存在的问题提供新的思路和工具，将产生深远的社会意义和效益。