视频中事件的内容分析与语义描述

Content analysis and semantic description of events from videos has wide applications on many fields such as intelligent video surveillance and video content retrieval. Most existing methods focus on relative simple events with short continuing time and simple movements, and very few of works are done on the analysis and description of long-time complex events with many complex motions and actions in videos. With the aim of inferring the "what of event, which of atomic events, and when of atomic events" semantic description, this proposal proposes a novel framework for analyzing video events which combines multiple computational levels such as low-level video primitive feature、mid-level atomic event and high-level event semantic description. Under this framework, we first investigate a common generative model to describe the two types of video primitives in a unified form: structural video primitives and textural video primitives. Then we exploit the automatic generation of a complete set of mid-level atomic events, and also focus on the automatic atomic event annotation of a large number of event videos. Finally, we will introduce a unified discriminative framework to jointly model the temporal segmentation of atomic events and semantic description of the overall event. This proposal poses significant importance on proposing new theroies and methodologies on video event analysis and also beneficial for developing more technologies and methods for wide applications of event analysis.

视频中事件的内容分析与语义描述是视频理解的核心内容，在智能视频监控、智能视频检索等领域有着广泛的应用前景。目前大部分的工作主要集中在持续时间较短、变化较少的事件识别，对持续时间较长、变化复杂的事件识别研究较少。本项目主要研究视频中复杂事件的内容分析与语义描述，建立在底层视觉模式、中层原子事件、高层事件语义等多个层次进行分析的计算方法，以获取"发生了什么事件、事件分为几个阶段、每个阶段分别出现了什么行为"的语义描述。研究内容包括：视频特征生成式表示与建模的理论和方法，分析视频中的基本视觉特征：结构基元和纹理基元；完备的原子事件集合的生成方法，并基于原子事件集合对视频中的原子事件类别进行标注；将原子事件的时序分割与整体事件的语义描述联合建模，得到统一的计算模型，进而建立高层语义的推理算法。

视频中事件的内容分析与语义描述是视频理解的核心内容，在智能视频监控、 智能视频检索等领域有着广泛的应用前景。目前大部分的工作主要集中在持续时间较短、变 化较少的事件识别，对持续时间较长、变化复杂的事件识别研究较少。本项目主要研究视频 中复杂事件的内容分析与语义描述，建立在底层视觉模式、中层原子事件、高层事件语义等多个层次进行分析的计算方法，以获取“发生了什么事件、事件分为几个阶段、每个阶段分别出现了什么行为”的语义描述。主要完成了层级视频描述的复杂行为理解方法，基于关键片段提取的视频事件检测方法，采用分解动作-场景网络的动作识别方法，采用三维运动图网络的动作识别方法，基于异质判别分析的跨视角动作识别方法以及基于多分量可变形部件模型的行人跟踪方法。在进行理论和算法研究的同时，进行了应用研究和工程实现，取得了良好的效果。