基于视觉场景理解的矢量地图与实景图像实时融合

The AR-based navigation has become increasingly important in location-based services, by overcoming the limitation of traditional 2D/3D maps, in terms of both service pattern and application. However, current AR-based navigation depends on the high-accuracy GPS localization and other high-cost infrastructure, and therefore cannot combine vector map and images for integral visualization. Targeting this, we develop a novel scene understanding based approach that matches the vector map and real world images to provide online and offline map service. Based on trajectory measure provided by smart phones inertial sensors, this approach captures and analyzes the real world image of the scene through smart phone camera. By real-time recognizing and extracting the semantic object and layout of the scene (geometric, semantic and topologic features), the vector map can fast register to the image through geometric and semantic constraints, and therefore realize the integral rendering of 2D vector map and real world images. The proposed approach provides the user perspective consistent AR rendering, and significantly improve the applicability and user experience of AR-based navigation.

基于增强现实(AR)的导航与位置服务技术能够克服传统2D/3D地图服务在导航模式和应用上局限性，是创新研究的突出热点。然而，当前的AR导航还过度依赖高精度的GPS定位和高成本的元器件，无法将矢量地图与实景拍摄图像进行深度融合。对此，本课题拟发展一种基于视觉场景理解的矢量地图与实景图像实时融合新方法，以网络在线或者离线地图服务为基础，结合智能移动终端惯性测量传感器，通过对智能移动终端摄像头获取的实景图像进行视觉理解，实时识别并且提取与地理场景认知和地图要素布局相关的特征信息（如几何、语义、拓扑等信息）,与地图快速匹配，建立实景图像与地图在空间位置与要素上的多层次关联；利用几何与语义一致性约束，实时将矢量地图信息从空间坐标系转换为实景图像坐标系，实现二维矢量地图与实景图像融合渲染，提供与用户观察视角一致的增强现实AR视图，从而极大提升AR导航的可用性与便捷性。

基于增强现实(AR)的导航与位置服务技术能够克服传统2D/3D地图服务在导航模式和应用上局限性，是创新研究的突出热点。然而，当前的AR导航还过度依赖高精度的GPS定位和高成本的元器件，无法将矢量地图与实景拍摄图像进行深度融合。对此，本课题发展了一种基于视觉场景理解的矢量地图与实景图像实时融合新方法，以网络在线或者离线地图服务为基础，结合智能移动终端惯性测量传感器，通过对智能移动终端摄像头获取的实景图像进行视觉理解，实时识别并且提取与地理场景认知和地图要素布局相关的特征信息（如几何、语义、拓扑等信息）,与地图快速匹配，建立实景图像与地图在空间位置与要素上的多层次关联；利用几何与语义一致性约束，实时将矢量地图信息从空间坐标系转换为实景图像坐标系，实现二维矢量地图与实景图像融合渲染，提供与用户观察视角一致的增强现实AR视图，从而极大提升AR导航的可用性与便捷性。