无标记运动捕捉方法与理论

Markerless motion capture is an appealing solution for digitally recording the biologic motion information with visual signals. It is one of the hottest topics in the realms of computer vision, image processing and computer graphics, etc. It plays an important role in many fields, such as culture, military, education, medical treatment, etc. Currently, there are two major challenges existing in the markerless motion capture: capture the high accuracy motion; achieve stable motion capture over time. In order to break the limit of precision and robustness, in the project, we will carry out the Key technology research for markerless motion capture: (1) For facial motion capture, an adaptive dynamic facial expression is constructed for the realtime performance and robust computations. The motion capture system jointly solves for a detailed 3D expression model of the user and the corresponding dynamic tracking parameters; (2) For hand manipulation motion capture, we introduces a composite motion control to simultaneously model hand articulation, object movement, and subtle interaction between the hand and object. An optimal motion control that drives the simulation to best match the observed image data is searched based on the contact information. Convenient and high accuracy motion capture and motion retargeting results can be obtained by the proposed system; (3) For uncontrolled background human body motion capture. The moving camera parameters and the dense depth maps are firstly recovered from the significant amount of dynamic pixels video streams. Then, a sparse constraint and a dynamic view dependent texture model is adopted for reducing the pose ambiguity. The proposed system can improve the stability of motion capture and motion retargeting in outdoor environment.

无标记运动捕捉通过视觉信号记录生物的运动信息，已成为计算机视觉、图像处理、计算机图形学等多学科交叉领域的研究热点，在文化、军事、教育、医疗等领域有着广泛的应用价值，主要面临着两大挑战：捕捉高精度的运动信息；实现运动信息的稳定捕捉。为了突破现有无标记运动捕捉精度和稳定度固有局限，本项目拟开展无标记运动捕捉关键技术研究：（1）针对脸部表情运动捕捉，建立自适应动态脸部表情模型，同步优化脸部基本表情及组合系数，解决计算时间复杂度与高精度运动捕捉的矛盾；（2）针对手与物体交互运动捕捉，建立手、物体以及相互接触的动力学方程，提出复合运动控制器模型，采用基于接触点采样的控制器优化方法，实现便捷和高精度交互运动的运动映射；（3）针对户外人体运动捕捉，通过恢复移动相机的空间位置，重建运动对象的稠密点云，建立运动稀疏性约束与运动对象的动态纹理模型，提高室外无标记运动捕捉与运动映射的稳定性。

无标记运动捕捉通过视觉信号记录物体尤其是生物的运动信息，已成为计算机视觉、图像处理、计算机图形学等多学科交叉领域的研究热点，在文化、军事、教育、医疗等领域有着广泛的应用价值，主要面临着两大挑战：捕捉高精度的运动信息；实现运动信息的稳定捕捉。为了突破现有无标记运动捕捉精度和稳定度固有局限，本项目开展了无标记运动捕捉的关键技术研究：（1）针对脸部表情运动捕捉本项目建立了自适应动态脸部表情模型，同步优化脸部基本表情及组合系数，解决了计算时间复杂度与高精度运动捕捉的矛盾；（2）针对手与物体交互运动捕捉，本项目建立了手、物体以及相互接触的动力学方程，提出了复合运动控制器模型，采用基于接触点采样的控制器优化方法，实现了便捷和高精度交互运动的运动映射；（3）针对户内外快速相机移动下的人体运动捕捉，项目通过多传感器融合，建立了快速相机运动下的特征跟踪模型，在估计快速移动的相机的三维空间位置的同时，恢复场景的稠密点云，建立运动稀疏性约束与运动对象的动态纹理模型,提高了无标记运动捕捉与运动映射的稳定性。