面向智能汽车的深度多光谱融合行人检测方法研究

Pedestrian detection in the complex traffic environment is a key technology to improve the ability of intelligent autonomous driving vehicle for perceiving pedestrian. We will study the pedestrian detection based on multispectral (visible and infrared) image fusion. This project aims at proposing novel pedestrian detection method to address the major challenges of pedestrian detection, i.e., partial occlusions, nighttime without sufficient illumination problems as follows: (1) Building a novel multi-scale full convolution generative adversarial network based infrared thermal images generation form visible spectrum images framework so as to provide the label data for occluded pedestrian detection based on visible and infrared fusion; (2) Researching multi-scale deep unsupervised network model based visible and infrared image fusion. By this way, the bottleneck problem of the requirement of the input images with high registration accuracy can be broken. (3) Constructing novel multi-scale semantic self-attention deep learning network fusion to detect the pedestrians, so as to improve the detection accuracy in all-time. The proposed method will be applied in intelligent autonomous driving vehicles, so as to improve the perception ability of autonomous driving system for the surrounding traffic environment, and provide theoretical and technical supports for the high intelligence of intelligent vehicles.

针对智能汽车自动驾驶系统对周围复杂交通环境中行人检测的问题，本项目拟从基于多光谱(可见光与红外图像)融合行人检测的角度出发，提出针对复杂环境中被遮挡行人检测的新方法。研究内容包括：(1) 提出基于多尺度全卷积生成对抗网络模型的可见光图像生成红外图像方法, 为实现基于可见光与红外图像融合的被遮挡行人检测提供标签数据；(2) 提出基于多任务深度无监督网络的可见光与红外图像融合方法，突破现有红外图像与可见光图像融合需要输入两幅高精度配准图像的瓶颈；(3)提出基于多尺度语义自感知融合的深度行人全天时检测方法，有效提升不同光照环境下被遮挡行人检测的精度。本项目研究成果将应用于提高智能汽车自动驾驶系统对周围交通环境中行人的感知能力，为我国智能汽车的高度智能化提供理论与技术支撑。

本项目针对智能汽车自动驾驶对周围复杂交通环境中行人检测的难题，重点研究了如何利用多光谱融合图像提高行人特征的辨识度：提出了基于生成对抗学习的可见光与红外图像融合的新方法，为实现基于可见光与红外图像融合的被遮挡行人检测提供数据保障；一种基于多任务深度无监督网络的可见光与红外图像融合方法，突破现有红外图像与可见光图像融合需要输入两幅高精度配准图像的瓶颈，获得了良好的融合图像；基于融合图像，本项目提出基于多尺度语义自感知融合的深度行人全天时检测方法，有效提升了不同光照环境下被遮挡行人检测的精度。在该项目的资助下，目前发表学术论文9篇，其中SCI检索论文7篇，申请国家发明专利9项；培养博士2名，硕士生5名。