空-地机器人跨域协同视觉定位与建图问题研究

Aerial-ground robots have very different visual perspective to environments, which leads to big challenges to visual data association in aerial-ground cross-domain cooperative visual localization and mapping. In order to solve this challenge, this project proposes novel ideas and methods by introducing recent deep learning method into visual simultaneous localization and mapping (SLAM), aerial-ground cross-domain place recognition and target three-dimensional (3D) model prediction. ..1) Datasets for aerial-ground cross-domain cooperative visual localization and mapping is created by using both 3D simulation data and real-world data, which can fulfill the requirements of training deep neural networks (DNNs) in this project...2) Semantic dense map is created in the visual SLAM system by integrating a DNN in the visual SLAM process for semantic analysis of the map...3) A novel aerial-ground cross-domain place recognition method is proposed. A DNN-based method is used to achieve aerial-ground image association...4) A novel aerial-ground cross-domain cooperative visual SLAM framework and algorithm is proposed. 3D models of semantic targets in the environment is predicted by a DNN, which supports the error model of map matching between the aerial and ground semantic dense map... The results of this projects will facilitate both theoretical and technical foundations for aerial-ground cross-domain cooperation applications of unmanned aerial vehicles and unmanned ground vehicles.

本项目针对空-地机器人跨域协同视觉定位与建图中面临的跨域大视角差异条件下空-地视觉数据关联的挑战，将深度学习方法的最新研究成果引入到视觉同步定位与建图(SLAM)、空-地跨域地点识别、目标三维模型预测中，提出应对该挑战的新思路和新方法。1、提出以三维模拟数据与实际环境数据相结合的方式构建空-地协同视觉定位与建图数据集，满足相关深度神经网络训练需要；2、基于深度神经网络在稠密地图视觉SLAM过程中进行地图语义分析，实现语义稠密地图构建；3、提出基于深度神经网络的空-地跨域地点识别方法，实现空-地图像的数据关联；4、提出空-地跨域协同视觉SLAM框架及算法，基于深度神经网络预测环境中语义目标的三维模型，并以此为基础结合语义稠密地图构建空-地跨域地图匹配误差模型，实现空-地跨域协同视觉SLAM。本课题研究成果能为无人机、无人车空-地跨域协同的实际应用奠定一定的理论基础并提供技术支撑。

本项目针对空-地机器人跨域协同视觉定位与建图中面临的跨域大视角差异条件下空-地视觉数据关联的挑战，围绕“三维环境视觉感知信息的抽象机理”、“空-地跨域大视角差异条件下地图数据的融合”科学问题开展关键技术研究，在基于异构传感器数据的三维地图匹配、三维位姿估计、环境中物体的位姿估计、环境中物体的三维表面模型估计、基于三维点云的空-地机器人跨域协同定位等方面取得了关键技术研究进展，一定程度上实现了空-地跨域协同SLAM。本课题研究成果能为无人机、无人车空-地跨域协同的实际应用奠定一定的理论基础并提供技术支撑。