多视点视频自适应压缩感知与并行多级预测重构研究

The multi-view video (MVV) is widely applied in fields including 3DTV, free viewpoint video, tele-medicine clinics, wireless multimedia sensor networks and surveillance systems. With lower cost, more energy-efficiency, and less requirement in bandwidth and memory, the image sensor based on compressed sensing (CS) is an effective solution to the problem of storage and transmission for excessively voluminous data in MVV system. However, the available research for MVV system with CS which is based on non-adaptive compression projection, results in reconstruction with a high computational complexity and a low accuracy, which is infeasible to the application of non-stationary MVV. Although the adaptive CS researches based on Bayesian theory, sparse model approximation, and adaptive blocking sampling are developed, these method cannot be used for realtime implementation in application of compressive video. Therefore, the adaptive compressive sampling and parallel reconstruction by multi-level prediction of MVV system are proposed in the project. Firstly, the method adjusts the compressive sampling rate and frame rate based on the detection and estimation for sparsity and motion features of scene content, and achieves the adaptive compressive sampling for MVV. The compressive samples are then encoded with a high efficiency for transmission. Based on the adaptive side information generated by spatial and temporal correlation, the fast parallel reconstruction for MVV with a high precision by multi-level prediction. And finally, a set of theories and methods for adaptive compressed sensing and exact reconstruction for MVV are contributed with a proprietary intellectual property rights.

多视点视频（MVV）在三维电视、自由视点视频、远程医学诊疗、无线多媒体传感器网络、视频监控系统等领域有重要应用。压缩感知（CS）图像传感器成本低、能效高、存储和带宽需求低，是解决MVV系统海量视频存储与传输瓶颈的有效方法。然而， 现有的MVV压缩感知研究基于非自适应压缩投影，重构复杂度高、且精度低，不适合非平稳MVV应用。尽管发展了基于贝叶斯感知、稀疏性模型逼近和自适应分块采样等感知策略，但这些方法不能用于视频实时压缩采样。据此提出MVV的自适应压缩感知和并行多级预测重构研究。该研究基于场景内容的稀疏性和运动特性自适应调整MVV的压缩采样率和帧速率，实现MVV自适应压缩感知；解码端利用时间和空间相关性产生自适应边信息，通过并行多级预测重构处理，实现MVV的高精度和低复杂度重构。最终获得一套具有自主知识产权的MVV自适应压缩感知与快速高精度重构理论与方法。

3D视频采集是视频处理和通信领域的研究热点问题之一。多视点视频(MVV) 是一种有效的3D视频表示方法，它能更生动地再现3D场景，提供立体感知和交互功能。MVV数据量随着摄像机数目增加而线性增加，其海量视频数据的存储和传输，是制约MVV广泛应用的瓶颈。常规多视点编码（MVC）是在H.264/AVC预测编码基础上产生的，MVC编码器复杂、解码器简单，只适用于立体电视和视频点播等。多视点分布式视频编码（MDVC）在一定程度上改善了编码效率、速率失真性能和解码效率，但MDVC的编解码性能与H.264/AVC还存在较大差距、而且MDVC系统联合解码算法比MVC解码更复杂。此外，多视点分布式压缩感知(MDCS) 系统联合重构的计算复杂度高、重构精度低。.本项目研究内容主要包括两个方面。首先，针对常规MVV压缩采样研究过程中使用固定的变换基函数和测量矩阵进行压缩测量、不适用于非平稳MVV场景内容和影响重构精度的问题，采用了DSP硬件模块实时检测和定量估算MVV场景内容的稀疏性和帧间运动特性，提出并实现了压缩采样率的自适应控制和帧速率的自适应控制方法研究，实现了MVV自适应压缩采样。其次，针对常规MVV分布压缩感知系统解码端的计算复杂度高的问题，基于分布式压缩感知理论提出了多级并行预测解码方法。在解码端进行DPCM解码之后，通过MVV帧内结构预测、帧间运动估计和视点间视差估计获得自适应边信息（SI），采用多级自适应预测重构获得MVV高精度重构结果，还将重构算法并行化实现来提高了重构效率，有效降低了MVV解码复杂度，实现MVV高精度快速重构。.项目研究过程中解决了MVV自适应压缩采样和MVV快速高精度重构两个问题。MVV自适应压缩感知实现成本低，压缩采样之后的MVV海量视频数据得到了有效降维，其存储空间和带宽需求大大低，有效解决了MVV系统海量视频存储与传输的瓶颈问题。本项目的研究成果将在三维电视、自由视点视频、远程医学诊疗、无线多媒体传感器网络、视频监控系统等领域有重要意义。