三维视频的率失真优化编码技术研究

As three dimensional video (3DV) could provide real 3D sensations for users, it has attracted increasing attention in recent years. In order to be compatible with the current legacy devices and existing delivery facilities, multiview video plus depth representation has been extensively utilized to represent 3DV. Virtual views can be synthesized from the acquired muitiview videos and their corresponding depth maps, which is known as depth image-based rendering(DIBR) technology. Since the data volumes of 3DV are huge, efficient compression is a critical technology for the application of 3DV. Based on our previous work, virtual view oriented rate distortion optimization technologies will be researched so as to improve the coding efficiency of 3DV in this project. The research work contain three parts, i.e. accurate distrotion model of synthesized virtual views (especially the affection of depth coding error on synthesized virtual views), virtual view oriented Lagrange multiplier for depth maps, and joint bit allocation alogrithm between multiview videos and depth maps which can regulate the distortion difference between syntheiszed virtual views and multiview videos dynamically. This project will provide theoretical basis and novel implementation methods for rate distoriton optimization of 3DV; moreover, it will promote the standarization of 3DV coding to some extent.

三维视频(3DV)可以使用户享受到立体视觉感知，是近几年的研究热点。为了与已有视频编码标准兼容，3DV主要采用多视点视频-深度图表示。用户在3DV的终端可以采用基于深度-图像的虚拟视图生成技术得到其需要的虚拟视点的视频内容。3DV的数据量巨大，因此如何对3DV进行高效的压缩编码就成为3DV中的关键的技术之一。为提高3DV的编码性能，本项目拟在前期研究的基础上对三维视频编码的率失真优化技术进行深入研究。主要包括三个方面的工作：研究精确的虚拟视图失真模型，重点确定虚拟视图失真与深度图编码失真以及多视点视频编码失真之间的确切关系；研究确定适用于深度图的拉格朗日因子计算方法；设计虚拟视图失真与多视点视频失真差异可调节的多视点视频-深度图联合码率分配技术。本项目将为3DV的率失真优化技术提供理论依据与新的实现方法，并对3DV编码标准的制定起到一定的促进作用。

在本项目的资助下，项目组顺利开展了申请书中的相关研究内容。首先对虚拟视图合成过程进行了细致分析，提出一种快速准确地虚拟视图质量PSNR估计方法；其次对深度图的特征进行全面分析，提出了适用于深度图编码的拉格朗日因子计算方法和失真准则；最后结合人眼的主观感受，提出了由粗糙到精细的可调节的多视点纹理视频和深度图的联合码率分配算法，以及多视点纹理视频之间的比特分配算法。，除此之外，课题组与其他研究机构广泛合作，在研究基础上进行了扩展，主要包括码率控制算法，3DV处理算法，纹理视频编码预测算法，3DV中的视差估计算法等等。研究期间共计发表论文12篇，其中项目负责人以第一作者身份，在IEEE Transaction上发表论文4篇(1篇IEEE T CSVT, 1篇IEEE T MM, 2篇 IEEE T BC)。