层次化先验驱动的流形一致人脸超分辨率研究

The Face Super Resolution (FSR) based on manifold consistency assumption is severely compromised by the serious degradations (such as severe noise) when dealing with low-quality images (such as actual monitoring). In order to solve this problem, this project builds an FSR model driven guided by multi-level exemplar knowledge with bottom-up structure to enhance manifold consistency, starting with three levels of association mining for the exemplars: the pixel level, the block level, and the exemplar space level. ..First, we build the deformable content prior knowledge to improve the density of effective exemplar, enhancing the manifold consistency destructed by severe exemplar shortage...Second, we build the guidance of the context-sensitive prior knowledge and apply fine-grained analysis strategy to the contextual facial area, enhancing the manifold consistency destructed by the serious lack of block correlation...Finally, we build the reverse projection prior knowledge via multi-manifold for the guidance to conventional forward neighbor projection, enhancing the manifold consistency destructed by the inaccurate mapping...This project solves the manifold inconsistency problem through providing the guidance of three-layer knowledge, and thus solves the FSR problem of serious low-quality face image captured under practical scenarios represented by monitoring. It has important social and applicable value.

流形一致人脸超分辨率在处理低质量图像（如实际监控）时，因受严重降质干扰（如严重噪声），在底层特征、中层结构、高层投影的层面，产生底层特征不适配、细粒度人脸结构特征不明确、投影模型不准确的问题，导致流形一致假设遭严重破坏。为此，本项目构建自底向上的层次化样本知识增强流形一致性：首先，提出形变模式的知识表达结构，将具有严重降质差异的异质图像统一到同一特征层面，增强样本泛化能力，解决底层特征不适配问题；其次，揭示人脸精细上下文结构知识的关联机理，提出人脸有效结构补充的细粒度局部结构组合机制，解决人脸结构特征不明确的问题；最后，揭示反向投影的流形一致性映射规律，建立高阶近邻投影过渡模型，有效缓解降质导致的剧烈维度差，解决投影模型不准确的问题。本项目旨在通过融合多层知识协同增强流形一致性，解决严重低质人脸图像的超分辨率问题，适用于刑事侦查、人脸分析等关键技术领域，对保障人民生命财产安全具有重要意义。

本项目主要研究了流形一致人脸超分辨率在处理低质量图像（如实际监控）时，因受严重降质干扰（如严重噪声），在底层特征、中层结构、高层投影的层面，产生底层特征不适配、细粒度人脸结构特征不明确、投影模型不准确的问题，导致流形一致假设遭严重破坏。首先，提出形变模式的知识表达结构，将具有严重降质差异的异质图像统一到同一特征层面，增强样本泛化能力，解决底层特征不适配问题；其次，揭示人脸精细上下文结构知识的关联机理，提出人脸有效结构补充的细粒度局部结构组合机制，解决人脸结构特征不明确的问题；最后，揭示反向投影的流形一致性映射规律，建立高阶近邻投影过渡模型，有效缓解降质导致的剧烈维度差，解决投影模型不准确的问题。本项目旨在通过融合多层知识协同增强流形一致性，解决严重低质人脸图像的超分辨率问题，适用于刑事侦查、人脸分析等关键技术领域，对保障人民生命财产安全具有重要意义。在极低暗室条件下（人脸像素30×20，人脸照度1.5lux），PSNR和SSIM两个重要指标上分别获得2.0dB和0.15的提升,人脸识别率获得0.1的提升。2020年到2022年期间，在本项目的资助下，项目组成员共计发表论文9篇，其中SCI论文6篇，申请专利2项，培养研究生4人，完成了既定的发表和录用论文5-7篇的指标，圆满完成了项目目标。