融合多视点深度场的三维视频高效编码与精确重构

The processing of the three dimensional video with multi-view depth of field is the highlight topic in computer vision, image and video processing, and other academic fields. In this topic, the multi-view videos has a huge volume of data, depth of field is captured by depth sensors with relative low precision, and the visual field lacks of the accurate space information. These problems make a great challenge for high efficient coding and accurate reconstruction of the three dimensional video. In order to handle this challenge, following techniques need concerns, including depth of field accuracy optimization, joint high efficient coding on multi-view depth videos and color videos, high accuracy reconstruction of original captured scene with quantized depth and video. In our project research proposal, we focus on solving the above three techniques as following. Firstly, we will optimize and improve the accuracy of depth of field by modeling the geometric relationship between multi-view depth and video, and a global optimization model for temporal-spatial constraint risk estimation. Then, we will exploit the high performance prediction scheme based on high accuracy depth for multi-view geometric mapping. This scheme will improve the coding performance for multi-view depth videos and color videos. Finally, we setup a confidence evaluation model for decoded and quantized depth and color videos. This model is applied for high accurate reconstruction for captured original scene with inaccurate information. Based on the research of our project, the efficiency of multi-view depth of field based video coding and the accuracy of the reconstruction for three dimensional video will be promoted greatly. The achievements of our researches will benefit for three dimensional video both in the literature and in industry.

融合多视点深度场的三维视频技术是当前国际研究热点，涉及计算机视觉、图像视频处理等多个学科领域的交叉。多视点数据量庞大，深度场精度受限，视觉场缺乏空间机理的精确刻画，给三维视频的高效压缩与精确重建带来极大挑战。如何精确刻画场景空间机理、深度发掘视觉场与深度场内在关联冗余、以及精确重构编码失真条件下的三维场景是前沿挑战难题，涉及核心问题包括深度场精度优化、深度场与视觉场联合编码、场景优化重构等。本项目针对三维视频多视点深度场与视觉场间内在关联特性，建立时空约束的风险评估全局优化模型，探索深度场精度提升方法；通过高精度深度场建立多视点几何映射的高效预测模型，提高三维视频深度场与视觉场的压缩性能；构建解码信息置信度模型，形成数据失真条件下的场景精确重建，实现融合多视点深度场的三维视频高效压缩与精确重构。本项目可实现多视点深度场与视觉场处理及重构的理论创新与技术突破，促进三维视频的广泛应用。

融合多视点深度场的三维视频技术是当前国际研究热点，涉及计算机视觉、图像视频处理等多个学科领域的交叉。多视点数据量庞大，深度场精度受限，视觉场缺乏空间机理的精确刻画，给三维视频的高效压缩与精确重建带来极大挑战。本项目针对三维视频多视点深度场与视觉场间内在关联特性，建立时-空-视点域联合深度场精度提升方法；通过高精度深度场建立高精度的三维视频深度场与视觉场的压缩算法，重点提升物体边缘结构等关键点的压缩质量；构建快速虚拟视点绘制算法，形成数据失真条件下的场景精确重建。本项目实现多视点深度场与视觉场处理及重构的理论创新与技术突破，可促进三维视频的广泛应用。