激光点云三维目标信息提取与视频图像融合的多目标跟踪算法研究

Problems such as complexity of object movement and lack of object observation information due to occlusion become the bottleneck which restricts the progress of multi-object tracking research. In view of the superiority of three-dimensional laser point cloud and the complementarity of video image, the fusion of the two can effectively improve accuracy and robustness of the tracking algorithm, and break through the limitations caused by single video tracking. Through excavating object floor characteristics of mass laser point cloud interpretation, this project studies laser point cloud multi-object detection of multi-layer joint features, and provides a new method for rapid extraction of object three-dimensional information. According to anisotropy of object movement, motion behavior model of the object is analyzed, and interaction between the object and environment is quantified into a variety of social forces. The influence of different factors on the object state change is described comprehensively to establish a more effective and efficient state transition model. Three-dimensional information of laser point cloud and feature information of video image are combined to express and judge the observed image from different levels, and the advantages of complementarity are effectively realized to improve robustness of object observation. Research results of the project not only reduce false detection due to that laser point cloud lacks of feature information, but also solve the problem that the occlusion is difficult to track due to the lack of depth information in video image, so as to provide new research ideas and theoretical basis for the improvement of multi-object tracking algorithm performance.

目标运动的复杂性以及因遮挡造成目标观测信息缺失等问题，成为制约多目标跟踪研究进展的瓶颈。鉴于三维激光点云的优越性以及与视频图像的互补性，通过两者融合可有效提升跟踪算法的精度和鲁棒性，突破单视频跟踪所带来的局限。本项目通过对海量激光点云解译的目标底层特性进行挖掘，研究多层联合特征的激光点云多目标检测，为目标三维信息的快速提取提供新的方法；针对目标运动的各向异性，分析跟踪目标的运动行为模式，将目标与环境间的交互量化为多种社会力，综合描述不同因素对目标状态变化的影响，建立更加有效、高效的状态转移模型；融合激光点云的三维信息和视频图像的特征信息，从不同层面对观测图像进行表达和判别，有效的实现优势互补，提高对目标观测的鲁棒性。项目研究成果既降低激光点云因缺乏特征信息造成一定误检测的问题，又解决视频图像中因缺少深度信息易出现遮挡难以跟踪的问题，从而为多目标跟踪算法性能的提升提供新的研究思路和理论依据。

目标运动的复杂性以及因遮挡造成目标观测信息缺失等问题，成为制约多目标跟踪研究进展的瓶颈。三维激光点云与视频图像的融合可有效提升多目标跟踪算法的精度和鲁棒性，突破单视频跟踪所带来的局限。本项目分析激光点云三维数据特性，通过深度特征提取网络，分别获取激光点云和视频图像的深度特征，使用双流网络持续融合两种视图的深度特征，并根据目标检测中不同的检测任务，设计多任务多损失的目标检测器，能够实现准确的目标检测，尤其对遮挡目标和小目标检测具有一定的优势，可应用于无人驾驶场景中。在视频图像的目标检测研究中，从主干网特征提取、图像去噪、目标变化等角度考虑，提出了三种主要的目标检测方法：全局特征增强与自适应回归的目标检测方法、图像压缩噪声去除的目标检测方法、两阶段深度网络的目标检测方法，检测精度均优于同类相关的主流检测方法，已将研究提出的检测方法应用于输电线路异常目标检测、银行卡号识别、自助柜商品识别中。在多目标跟踪的研究中，通过优化目标检测、特征提取、关联性分析三个阶段，提升视频多目标跟踪的整体性能，提出深度特征提取及关联性分析的多目标跟踪方法，该方法较好的解决了目标频繁遮挡、跟踪目标进出视频帧的问题，在多目标跟踪公共基准数据集MOT15、MOT16、MOT17上进行了详细的实验，优于目前流行的多目标跟踪方法。