基于RGBD相机的实时三维建模技术研究

In recent years, the rapid development and popularization of RGBD cameras provide ordinary users with a convenient and fast way for acquiring 3D data, which makes the 3D scanning and modeling using RGBD cameras a hot research topic in both academic and industrial circles. Most existing 3D modeling techniques are only suitable for offline computation, and the study of realtime scanning and modeling is still at the initial stage, unable to meet the requirements of realtime applications using RGBD cameras. In this project, we propose to study realtime 3D modeling techniques based on RGBD cameras, which aims to overcome the limitations of existing 3D modeling techniques in terms of realtime capability, robustness and universality, and from the three aspects of data structure, algorithm and system develop an innovative and practically useful realtime 3D modeling system. The research content of the project mainly includes online structure analysis and realtime completion of the 3D scanning data of individual objects, realtime registration of color texture and 3D scanning data, online structure analysis and realtime reconstruction of large-cale 3D scenes, and simple interactive modeling based on RGBD cameras. The research results of the project will promote the development of film and television animation, games, engineering design and simulation, and has great impact in the fields of electronic commerce, national security, 3D big data, and etc.

近年来RGBD相机的快速发展和普及为普通用户提供了方便快捷的三维数据获取手段，这使得采用RGBD相机的三维扫描和建模成为当前学术界和工业界的研究热点。现有的三维建模技术大都只适合于离线计算，而实时扫描和建模的研究还处于初期阶段，无法满足RGBD相机实时应用需求。本项目提出研究基于RGBD相机的实时三维建模技术，旨在突破现有三维建模技术在实时性、鲁棒性和普适性方面的局限性，从数据结构、算法和系统三个层面研制出兼具创新性和实用性的实时三维建模系统。项目的研究内容主要包括单个物体三维扫描数据的在线结构分析与实时补全、颜色纹理与三维扫描数据的实时配准、大规模三维场景的在线结构分析与实时重建、以及基于RGBD相机的简易交互建模。项目的研究成果将促进影视动漫、游戏、工程设计与仿真等产业的发展，并对电子商务、国防安全、以及3D大数据等领域具有十分重要意义。

三维建模是计算机图形学的核心研究内容。传统三维建模方法和软件依赖于繁琐的手工操作，或需要使用专业设备来获取充足的三维信息。这些软件和设备只适用于专业应用环境，建模效率低、成本高，无法在互联网等新兴应用中普及使用。为提高建模效率、降低建模成本，项目探索了使用轻量级设备（如深度相机、智能手机）获取的数据来构建高质量三维模型的新途径。其主要挑战体现在：首先，输入数据为2D或2.5D，缺乏对象的完整三维信息；其次，消费级应用中普通用户只能进行简单手工操作，希望全自动建模。项目围绕着上述挑战，开展了基于RGB/RGBD图像的三维建模研究，提出了基于RGB/RGBD图像的三维头发全自动建模方法、基于单幅RGB图像的三维人脸实时动画方法、基于RGB图像的数字化身构建方法、基于RGB图像的机械零件建模方法、以及三维形状的深度学习方法。在计算机图形学顶级期刊ACM TOG和IEEE TVCG、Computer Graphics Forum上发表6篇论文，获得4项发明专利。部分专利技术已授权给欧莱雅和宝洁公司。