增强现实中的动态可见性计算研究

In augmented reality, the task of visibility computation is to determine the visibility of the virtual and real scenes with respect to the viewpoint and the light source, which then can be used for occlusion handling and shadow casting, therefore is the fundation of real-virtual composite. Existing methods for visibility computation are limited in the following two aspects: first, lacking the ability to deal with dynamic scenes online in real-time, as the result can not be adopted by most AR systems; second, can not guarantee accuracy and robustness of the results, therefore is difficult to achieve high quality. To solve the above problems, at the same in consideration of tendency of hardware development, we plan to further study visibility computation by exploring depth information, and to seek for an unified approach to compute the visibility of viewpoint and light source. The main issue of depth-based methods is to robustly deal with the errors and noise of the depth map, as well as the under-constrained problem due to the missing of 3D model, for which purpose two key problems should be addressed, one is the reliable segmentation and visual continuity optimization of dynamic spatial-temporal video regions, the other is the online local depth fusion and reconstruction. We will focus on solving the above two problems in this project, the achievements also can promote other problems in augmented reality.

增强现实可见性计算主要是要确定虚实场景与视点、光源之间的可见性关系，以进行虚实遮挡处理并合成阴影，这是正确进行虚实合成的基础。现有的可见性计算方法主要存在两点局限性：一是缺乏对动态场景和实时在线需求的针对性，在实际应用时受到很大限制；二是不能保证结果的精度和可靠性，难以获得高品质的融合效果。针对上述问题，并结合当前硬件的发展趋势，本项目拟采用基于深度的基本方法，对增强现实的可见性计算问题进行系统、深入的研究，形成处理视点和光源可见性问题的统一理论和有效方法。基于深度的方法主要需解决深度图中错误、噪声的影响，以及欠约束的困难。为此需解决两个关键问题：一是动态视频时空区域的可信分割和视觉连续性优化；二是动态场景的在线局部深度融合与重建。本项目的研究内容将围绕上述两个问题展开，研究成果对增强现实中的其它问题也具有很重要的促进作用。

增强现实中的虚实物体之间的可见性计算是保证正确进行虚实融合的基础。本项目主要研究与增强现实可见性计算相关的三个关键问题，即真实物体对虚拟物体的遮挡处理、虚拟物体在真实场景中的阴影区域计算、以及真实景物在虚拟物体上的投影区域计算。针对这三个问题，我们研究了基于颜色和深度的精确视频分割、深度图增强以及基于视频的场景光照分析和运动一致的视频融合等核心关键技术，并在此基础上构建了增强现实可见性计算的原型系统。在图形学顶级期刊ACM Trans. on Graphics上发表论文2篇，其它SCI期刊论文3篇，EI论文2篇，申请并获得发明专利授权1项。研究成果对实现高真实感的增强现实虚实融合具有基础性的重要作用。