广域低帧率航拍场景下在线目标跟踪算法研究

Online object tracking in wide area and low frame rate aerial video sequences has important research significance and application value for many fields such as military reconnaissance, city surveillance and so on. However, it is a very difficult task to achieve robust tracking in this scenario due to small object and low frame rate, and there also exist no large-scale evaluation benchmark and baseline algorithm in this filed. This project will propose deep object appearance models to depict the appearance information of small objects and temporal-spatial context association models to describe the motion information of similar objects in the low frame rate conditions, and will use the idea of local multiple object tracking to solve the online single object tracking problem in aerial scenarios. First, we will combine feature classification and Siamese matching deep networks to model object appearance and further exploit a cascade framework to integrate the shallow and deep modules to describe object appearance in a fast and robust manner.Second, we will simulate small object samples and aerial video frames to obtain large numbers of virtual training samples by using generative adversarial nets, and will adopt a step-by-step training strategy to train the deep neural networks effectively. In addition, we will associate the motion information of similar objects by introducing the idea of multiple object tracking in a local temporal spatial region, combine deep object appearance and temporal-spatial context information within similar objects, and therefore build the temporal-spatial context association model to infer the optimal trajectory of the tracked object. The ultimate goal of this project will build the large-scale evaluation benchmark and baseline algorithms for this research topic, make significant progress on the large-scale evaluation, and support the theoretical study and technology advance of the online object tracking problem in wide area and low frame rate aerial video sequences.

广域低帧率航拍视频序列中的在线目标跟踪在军事侦察、城市监控等方面具有重要研究意义和应用价值。然而小目标和低帧率使得该场景下实现鲁棒跟踪异常困难，且本领域缺乏大尺度测评数据库和基准算法。本项目拟提出针对小目标外观建模的深度目标外观模型和解决低帧率情况相似目标运动建模的时空语义关联模型，从而利用局部多目标跟踪思想解决该场景下的在线单目标跟踪问题。拟融合特征分类和孪生匹配深度网络进行目标外观建模，并采用级联浅层学习与深度学习架构快速精确刻画目标外观；拟利用对抗生成网络模拟小目标样本和航拍视频帧以获得大量虚拟训练样本，并尝试层级训练策略实现深度网络的有效训练；拟引入局部时空区域内多目标跟踪思想关联相似目标的运动信息，结合目标深度外观模型和相似目标间运动语义信息，建立时空语义关联模型推理被跟踪目标的最优轨迹。本项目拟最终建立该领域大尺度数据库和原型算法，为该场景中目标跟踪理论研究及技术进步提供依据。

航拍视频序列中的在线目标跟踪在军事侦察、城市监控等方面具有重要研究意义和应用价值。本项目主要从深度模型和深浅层模型融合的角度较为系统地研究了在线目标跟踪问题并在航拍视频序列上进行验证。在研究过程中，项目团队提出了自适应空间正则相关滤波（CVPR2019）、“略读-精度”长时跟踪框架（ICCV2019）、“元更新器”长时跟踪模型（CVPR2020，论文获奖提名）、深度互学习跟踪模型（PR2021）、Transformer跟踪模型（CVPR2021）等有效的跟踪模型及算法，在跟踪精度和速度上均取得了较为突出的进展，在标准跟踪测评中达到领域前沿性能。在本项目资助下，课题组共发表论文14篇，包括TPAMI、IJCV、TIP、PR等顶级期刊论文4篇，CVPR、ICCV、ECCV等顶级会议论文8篇，ICASSP等权威会议论文2篇。基于上述部分成果，课题组获CCF自然科学二等奖1项、国际视觉目标跟踪竞赛冠军6次等国内外奖励，课题负责人获批国家自然科学基金优秀青年科学基金资助。