城市级大场景中快速鲁棒的相机定位方法研究

Camera localization refers to obtaining the 6-DOF camera pose relative to a 3D model, which is typically estimated from the correspondences between 2D points in the image and 3D points in the model.As the fast development of autonomous vehicles, intelligent robots and augmented reality technologies, the investigation of camera localization in city-scale outdoor scenes becomes more and more important. However, due to the enormous 3D model, viewpoint and day-night changes, weather variations etc., it is very difficult to obtain the 2D-3D correspondences both fast and accurately, and the existing methods are far from the requirements of practical usages on the robustness, the speed and accuracy. This project will focus on large scale 2D-3D matching which is the essential problem in camera localization,and aims to substantially improve its accuracy and efficiency through the development of better local features and indexing methods.The main contents are as follows: (1) improving the discriminability of local features by using convolutional neural networks to integrate the spatial context information from the entire image. (2) extracting viewpoint and illumination invariant local features by exploiting the fact that Generative Adversarial Networks can reconstruct images under some controlled factors. (3) using random decision forests to indexing the 3D points, and training the forests with supervisions in the database to improve the efficiency of 2D-3D matching.

相机定位指根据拍摄的图像获取相机相对于场景三维模型的6自由度姿态，通常可由图像二维点和模型三维点的对应关系来估计。随着自动驾驶、智能机器人、增强现实等技术的蓬勃发展，研究城市级室外大场景中的相机定位凸显出越来越重要的实际意义。然而由于模型庞大、视角差异、白-夜和天气变化等因素，快速准确地建立二维-三维对应关系难度很大，现有的定位方法在鲁棒性、速度和精度上都远未达到实际应用的要求。本项目将针对大规模二维-三维匹配这一核心问题进行研究，旨在以局部特征提取和三维点索引为突破口，实质性地提高匹配的准确性和速度，进而提升定位的鲁棒性、速度和精度。具体内容包括：(1)用卷积神经网络融合整体图像的空间上下文信息来提升局部特征的区分度(2)利用生成对抗网络可在受控条件下重构图像的特性提取具有视角和光照不变性的局部特征(3)采用随机决策森林对三维点进行有监督的索引，充分利用数据库中的监督信息提高匹配的效率。

相机定位指根据拍摄的图像获取相机相对于场景三维模型的6自由度姿态，由于自动驾驶、智能机器人、增强现实等技术的蓬勃发展和巨大的推广应用的前景，研究作为核心技术之一的城市级室外大场景中的相机定位有重要的意义。相机定位通常可由图像中二维点和模型三维点的对应关系来估计。本项目以提升2D-3D特征匹配的准确率和效率为目标进行研究，包括融合卷积网络中不同层级的空间上下文信息来提升增强特征的区分度，利用生成对抗式网络生成模拟图像并与原始图像的特征进行融合以提升特征的鲁棒性，合并全局特征与局部特征的提取网络降低特征提取耗时，用深度神经网络对满足几何一致性的2D-3D匹配进行筛选。实验验证融合多尺度特征以及融合生成图像的特征能够有效提升特征的区分度和鲁棒性，为学术界相关研究提供参考，全局特征与局部特征的合并能将特征提取速度提升1倍，2D-3D匹配网络能显著提升匹配准确率，从而提升相机定位的精度，有较高的实际应用价值。