基于层次化建模和联合任务学习的复杂行为与事件分析

To deal with large scale video data, automatic video data analysis techniques have drawn much attention. For complex video activities and events, this project studies the semantic gap, temporal structure modeling and representation learning in joint-task learning. We aim to propose hierarchical modeling and joint-task learning based video analysis theory and methods.. The research is divided into three layers. In the feature layer, combining with characteristics of complex activities and events, we summarize, compare and analyze existing low-level feature extracting methods, and study static and dynamic joint coding based on local and global features. In the model layer, we study the machine learning framework of video based probabilistic graphical models and deep learning models, and focus on three specific learning problems of semantic concept, temporal structure and task-driven representation. Besides, we establish video data integration and analysis platform to validate the proposed models. In the task layer, we study the relation among multiple machine learning tasks, including video classification, detection, annotation and retrieval. We build the unified learning framework of feature layer, model layer and task layer, and utilize supervisory information from multiple tasks to jointly learn enhanced video data representations.

为了应对大规模视频数据的处理问题，自动视频数据分析技术已经受到广泛的关注。本项目以复杂视频行为与事件为研究对象，针对语义鸿沟问题、时序建模问题和联合任务中表达学习等问题，研究基于层次化建模和联合任务学习的视频分析理论和方法。. 本项目把研究内容从低到高划分为三层。在特征层，结合视频数据（复杂行为和事件）的特点，归纳、总结、对比和分析现有的视频底层特征表达方法，研究基于局部和全局的静态和动态特征联合编码。在模型层，研究适用于视频分析的概率图模型和深度学习模型等机器学习框架，进而针对语义概念学习、时序结构学习和任务驱动的表达学习三个具体问题展开研究，并建立视频数据整合和分析平台来验证所提出的模型。在任务层，研究多个机器学习任务包括视频的分类、检测、标注和检索之间的相互联系，建立特征层、模型层和任务层统一学习框架，利用多个任务的监督信息联合学习增强型的视频数据表达。

本项目围绕复杂行为与事件分析研究方向，开展了层次化建模的理论模型研究与应用验证，在部件级行为识别、长跨度事件分析、场景语义理解等方面取得了一系列研究成果，突破了特征判别力低、时序关联弱、语义理解差等核心技术难题。发表学术论文24篇（包括TPAMI、IJCV等CCF-A类论文11篇）。相关技术与华为、百度等公司开展较多项目合作，取得一定的社会与经济效益。项目负责人入选中国科协青年人才托举工程和北京市科技新星计划。