面向3D-TOF相机的高质量深度图像重建方法研究

The ability to capture depth information of static real world objects has reached increased importance in many fields of application, such as: three-dimensional scene reconstruction, industrial robots, robot navigation. There exist a variety of depth measuring technologies to acquire depth information about our world. In general, they can be categorized into two major classes: passive range sensing methods and active range sensing methods. . Among passive range sensing approaches, stereo matching is probably the most well-known and most widely used method. Classical two-frame stereo matching computes a stereo disparity from a pair of images under given camera configuration. The key objective of stereo matching methods is to reduce the matching ambiguities introduced by low-texture regions, and can be generally classified into two categories: local methods and global methods. But unfortunately, neither local nor global approaches could solve the fundamental problems in stereo such as occlusion and low-texture regions. Hence, satisfactory depth results cannot be achieved by stereo matching approaches. Thus, depth maps from stereo matching for real scenes are often quite fragile. Compared to passive range sensing approaches, 3D Time-of-flight (3D-TOF) sensors use an active technique to obtain near real-time scene depth. These sensors measure time delay between transmission of a light pulse and detection of the reflected signal on an entire frame once by using extremely faster shutter. But in the current generation, these 3D-TOF sensors are limited in terms of resolution, which is typically less than 1/5th the resolution of a standard definition camera. . Thus, high-quality depth maps cannot be achieved by stereo matching approaches or 3D-TOF cameras. Motivated by above observations, investigating the benefits of stereo matching and 3D-TOF cameras, this project intends to construct a new depth map acquisition system. Given a low-resolution depth map from 3D-TOF, we recover a high-resolution high-quality depth map using the registered high-resolution color stereo image pair. First, based on the mutual benefits between raw depth map and features of high resolution color image, we model the relationship with two constraint terms of local and non-local priors and sufficiently explore their complementary nature. Moreover, we intend to build high-resolution depth map, high-resolution color image and the corresponding low resolution depth map sample libraries. By learning these sample libraries, we obtain the over-complete dictionaries to explore the inherent constraints between color and depth in the sample libraries. Furthermore, by considering reliable disparity pixels calculated from stereo matching algorithm, we intend to formulate a stereo disparity regularization term to further improve the quality of reconstructed depth maps. The proposed high-quality depth map acquisition system will have good prospects in many practical applications.

对场景深度图像的获取越来越受到大家的关注。目前，深度图像的获取手段主要有：立体匹配的方法和直接深度测量的方法。立体匹配算法在遮挡以及无纹理区域上无法匹配得到理想的深度图像，这一技术瓶颈导致立体匹配算法在实际应用中存在一定的局限性。直接进行深度观测的3D-TOF相机由于硬件系统的限制在分辨率上无法很好的满足实际应用的需求，从硬件系统方面进行改进也面临很难克服的技术难题。因此本项目拟结合立体匹配技术和3D-TOF相机各自的优势，构造一种新的深度图像重建系统，通过挖掘同场景彩色图像和深度图像在颜色和深度上的内在联系，同时结合左右彩色图像在视差和深度上的约束关系，提出高质量高分辨率深度图像的重建方法，并进而开发高质量高分辨率深度图像的实时获取系统。从而更好的为实际的应用提供支持，如：三维场景重建、工业机器人、机器人导航等。可以预见本项目拟构造的高质量深度图像重建系统将具有很好的应用前景。

场景的三维感知技术在移动机器人、虚拟现实、人机交互等场合都有很重要的应用，因此越来越受到大家的关注。目前，场景的三维感知方式主要有：立体视觉技术和直接深度测量技术。立体视觉在遮挡以及无纹理区域上无法匹配得到理想的视差图像，导致立体视觉技术在实际应用中存在一定的局限性。直接进行深度观测的3D-TOF相机由于硬件系统的限制分辨率比较低，无法很好的满足一些实际应用的需求。. 因此本项目结合立体视觉技术和3D-TOF相机各自的优势，开发了结合3D-TOF深度相机和左右彩色相机的融合视觉系统，重建了高质量高分辨率的场景深度图像。具体来说，通过本项目的研究，提出了一种高效的深度相机与彩色相机之间的配准方法；发掘了彩色图像和深度图像之间的约束关系，在高分辨率彩色图像的指导下开发了一种结合局部和非局部特征的深度图像高分辨率重建算法；结合立体匹配算法，提出了一种融合立体匹配视差结果的高质量深度图像重建算法；并开发了场景深度图像重建的软件处理平台。本项目开发的融合视觉系统可以很好的为三维场景重建、移动机器人导航等实际应用提供高质量的视觉支持。