几何纹理运动重建与数据驱动的真实感动画生成

Dynamic 3-D reconstruction and animation has wide application prospects in many fields such as battlefield sensing, medical，sports and entertainment. It provides full range of experience with variable viewpoint and movement，which brings people with stereoscopic visual、immersion and freshness. Current acquisition approaches and graphic methods cannot satisfy the requirement of applications: low cost, real-time estimation and strong sense of reality. Therefore, to achieve accurate and efficient dynamic 3-D reconstruction and animation with low-cost acquisition device has become a challenging problem. Grasping the opportunity of rapid development and application of Kinect camera, this project focuses on the research of data-driven photo-realistic animation based on 3-D reconstruction of geometry, texture and motion: based on sparse representation, explore the theory and method of acquiring, sensing and understanding of human motion scene; study the joint segmentation and sparse reconstruction of multi-view geometry and texture; explore the theory and method of real-time adaptive multi-priority motion estimation and deformation; investigate the law of continuously smooth change of dynamic characteristics; study the depth-guided animation generation method based on sparse representation. We are to reveal the representation mechanism of geometry, texture and motion in dynamic 3-D human scenes, set up a data-driven photo-realistic animation platform, and make breakthroughs in both theory and key technology.

动态三维重建与动画在军事、医疗、体育、娱乐等领域有着广泛的应用前景，其所提供的视角可变、运动可变的全方位感受带给人们视觉上的立体感、沉浸感和新鲜感。现有采集机制与图形学处理方法难以满足应用所需的“低成本”、“实时估计”、“真实感强”的特点，采用低成本采集设备实现精确高效的动态三维重建与动画是国际前沿挑战难题。本项目瞄准体感相机快速发展与应用的契机，开展几何纹理运动重建与数据驱动的真实感动画生成方法研究：以信号的稀疏表示为理论依据，探索采用一台体感相机对人体场景几何纹理运动进行采集、感知与理解的理论与方法。研究多视角几何纹理的联合分割与稀疏重建方法，探索实时的自适应多优先级运动估计及变形的理论与方法，发现动态特征的连续性平滑变化规律，研究基于稀疏表示的深度导向动画生成方法。力图揭示动态三维人体场景几何纹理运动特性的表示机理，建立数据驱动的真实感动画生成平台，在理论和关键技术研究上取得突破。

本项目采用一台第二代RGB-D体感相机采集同一个人的几何、纹理与运动信息，研究多视角几何纹理的联合分割与稀疏重建方法，探索实时的自适应多优先级运动估计及变形的理论与方法，以此为基础分析空时联合相关性，建立纹理信息稀疏重建模型，进而通过数据驱动实现逼真的新动作视频。共发表学术论文23篇，其中包括SCI期刊论文10篇，知名国际学术会议论文13篇；CCF-A类期刊2篇，CCF-B类期刊8篇，CCF-B类会议7篇，CCF-C类会议4篇。申请国家发明专利14项，已授权3项。在采用低成本便携设备实现高效采集、实时感知与动画方法等方面提出了原创方法，形成一套几何纹理运动重建与数据驱动的真实感动画生成核心技术，实现视角可变、运动可变的真实感动画效果。本项目所取得的成果是具有自主知识产权的技术，有望为我国在虚拟现实（VR）、增强现实（AR）技术的国际竞争中奠定技术基础。该成果可转化应用到战场环境重构、大规模场景视频监控、立体视频、影视游戏等领域，极大地促进我国信息技术与产业的发展，使我国在前沿领域占有一席之地，满足国家在建设和谐社会和发展国民经济方面的重大需求。