数据驱动的视频编码关键技术研究

Next-generation video coding is currently a hot research topic, involving digital TV broadcasting, security monitoring, mobile multimedia and other fields, which has important theoretical significance and application prospects. This project proposes to study the video coding methods with high efficiency and low complexity from the perspective of statistical machine learning, which focuses on three main aspects: firstly, according to the mode selection in video compression, data-driven method is proposed to extract the encoding features for intra-frame or inter-frame coding, and then the multi-task and multi-output representation learning method is proposed to quickly determine the encoding modes, which is able to save the computational complexity during rate-distortion optimization; secondly, convolutional neural network (CNN) is used to extract the associated encoding features of compression artifacts both from the "image-residual domain" and "pure residual domain", where the feature weights are proposed to be transmitted based on scene-change (or sequence video level), and the CNN-based methods are studied as the in-loop filter and post-processing modules to improve the compression efficiency; finally, considering the weights precision, model complexity and feature representation capacities of the related data-driven methods and the characteristics of current video coding framework, we propose to study the feature weights compression, integer weights modeling, and the feature representation learning based on human visual system so as to achieve the goal of using data-driven method to improve conventional block-based video coding techniques in depth.

高效视频编码是目前国际上的前沿热点，涉及数字电视、安防监控、移动多媒体等领域，具有重要的理论意义与应用前景。本项目将从统计机器学习角度研究高效低复杂度的视频编码方法，内容涉及三个方面：首先，针对视频编码过程的模式选择，提出利于数据驱动的方法提取帧内与帧间等编码特征，通过多任务多输出的特征表达方法快速决策编码模式，节省计算复杂度。其次，利用卷积神经网络从“图像－残差域”和“全残差域”的编码数据中联合提取视频压缩失真特征表达，同时研究以场景切换（或序列）为基本单位传输特征权重到编码端的策略，把基于卷积神经网络的失真恢复方法作为视频编码的环路滤波或帧后处理滤波模块，实现高质量视频编码。最后，考虑数据驱动方法的权重精度、复杂度、特征表达容量和传统视频编码框架的特点，通过研究特征表达权重的压缩、整数化方法，以及基于视觉特性的数据特征权重训练，实现数据驱动方法与视频编码技术的深度结合。

数据驱动的高效视频编码技术是数字电视、安防监控、移动多媒体等领域的共性关键技术，它具有重要的理论意义与应用前景。在此项目的支持下，我们针对基于数据驱动的编码模式预测策略、基于深度学习的失真恢复方法、在应用于视频编码的深度学习优化策略等三个方向展开了卓有成效的探索和研究。针对“下一代”超级块编码复杂度问题，我们提取了富有表达力的编码特征并设计了一种轻量级的神经网络用于编码模式预测，可节约平均20%以上的计算复杂度；针对多量化参数在编码复杂度高的问题，首次面向多量化参数编码方式提出了快速计算方式，编码复杂度可降低约50%；针对样点自适应补偿（SAO）滤波，首次提出了基于概率分布的样带补偿（BO）自适应调整方法，可以有效解决固定样带补偿的不足；针对多功能视频编码（VVC）压缩失真恢复，设计了一种包含时间融合网络和空间细节增强网络的端到端学习方案，在高量化参数情况下可平均提高0.30 dB；针对激光雷达点云压缩困难问题，提出了一种基于循环神经网络（RNN）的预测编码方法，效果好于现有的标准和商业软件。此外，我们也在三维视觉信号重建和处理方面做了一定的预研工作。以上研究成果，均发表于领域的顶级期刊或会议上。..总之，本项目的支持对于研究小组开展数据驱动的视频编码技术方面提供了不可替代的重要作用，为接下来开展智能视频编码技术奠定了基石。