基于多层分类结构和双目对比度感知的立体混合失真图像质量评价方法研究

Image quality has always been a golden rule to judge the performance of an image processing algorithm. It also serves as an important indicator to help select the best parameters when optimizing an image processing system. Thus, an objective image quality assessment (IQA) algorithm that operates based on simulating the most crucial properties of the human visual system (HVS) will be very valuable for developing the most advanced image technology. To enable more accurate quality assessment of multiply-distorted stereoscopic/3D images without the need for a reference, in this project, we propose a multi-layer classification framework trained on a large dataset to predict distortion parameter values of multiply-distorted images. First, a large 2D image dataset will be built by adding multiple distortion types and their combinations to the reference 2D images. After extracting the natural-scene-statistics features as the training data, a multi-layer classification-regression model will be trained to predict the five distortion parameter values of 2D images. Meanwhile, parametric functions will be learned to map the distortion parameter values to image quality, and the joint effect of different distortion types on image quality will be studied. Then, we will focus on solving the problem of how the HVS judges the quality of stereoscopic images; in particular, we aim to find a mathematical model that can effectively simulate the binocular fusion and rivalry process when an asymmetrically-distorted 3D image is viewed. Based on that model, the cyclopean images will be obtained by applying an optical flow algorithm to compute the disparity map, followed by an improved multipathway contrast gain-control model to compute the 3D mental view merged in human brain. Again, a large training dataset will be built by adding the same distortion types and combinations to the reference 3D images, and consequently, two multi-layer classification-regression models will be trained to predict the equivalent distortion parameter values of the cyclopean images. Finally, special techniques that effectively combine the quality degradations of the two monocular views and the cyclopean view will be studied, and a no-reference stereoscopic IQA approach that requires no training on human opinion scores will be proposed. Our research will establish a foundation to help monitor the quality of stereoscopic images in real applications.

图像质量是衡量图像处理算法性能优劣以及优化系统参数的重要指标，设计符合人眼视觉特性的客观质量评价方法对图像技术发展有重要的指导意义。为实现在无参考、混合失真情况下准确评价立体图像的质量，本项目拟提出构建大数据样本训练多层分类结构模型预测立体图像混合失真参数的策略。首先，根据不同失真的排列组合情况构建平面图像训练数据，提取自然图像统计特征，建立平面图像混合失真参数估计模型；重点分析失真参数与图像质量的关系，研究多种失真并存对图像质量的联合影响。其次，研究人类视觉系统对立体图像质量的感知规律，特别是非对称失真时双目融合竞争机制的建模方法；在此基础上利用光流算法和改进多通道对比度增益控制模型求解融合图像，构建立体图像训练数据，建立融合图像等效失真参数估计模型。最后，研究双目视觉质量加权组合方法，提出不依赖测试者主观评价的立体混合失真图像质量评价模型。本项目研究将为实际立体图像质量监测奠定基础。

随着3D技术应用领域的不断扩大，立体图像质量评价的研究越来越成为人们关注的焦点。图像质量是衡量图像处理算法性能优劣以及优化系统参数的重要指标，因此设计符合人眼视觉感知特性的客观质量评价方法对3D立体图像技术发展有着重要的指导意义，具有广阔的应用前景。为了进一步提高立体图像质量评价性能，特别是在无参考混合失真情况下，较准确合理的评价立体图像质量，本项目提出利用失真参数预测图像质量的策略。主要研究内容包括：1）2D图像混合失真类型和参数建模；2）双目立体视觉感知建模；3）融合图像混合失真参数建模求解；4）混合失真联合降质效应建模等。该项目的重要研究成果包括：1）提出用于混合失真参数建模的多层分类回归模型，该模型通过图像分类和概率加权方法，实现多失真参数预测；2）提出符合人眼视觉特性的自适应多通道对比度增益控制模型，该模型在空间域和变换域同时提取图像视觉质量特征，并在特征空间重构大脑融合图像，利用机器学习方法，实现对融合特征图像的失真参数估计；3）提出2D图像等效失真参数预测方法，并在此基础上构建训练数据，进而提出基于深度学习技术的2D图像混合失真参数快速预测网络模型；4）设计主观实验，通过数据分析建模和机器学习，提出用于融合图像失真参数估计的支持向量机模型；5）提出用于2D混合失真图像质量评价的最显著失真策略模型，以及针对非对称失真图像的双目图像质量融合的对比度权重策略；6）构建一个立体图像数据库，该数据库包含四种单失真和五种混合失真，以及每幅图像的真实失真参数，用于相关模型的训练和测试。实验结果表明，本项目提出的2D/3D图像质量评价方法，不受目前已有数据库的限制，且能保持较好的图像质量评价性能，因此具有更加广阔的应用前景和价值。本项目研究将为实际立体图像质量监测奠定基础。