基于多帧时空信息协同理解的自然感彩色红外成像方法研究

The natural color infrared imaging method is the research hotspot of infrared color night vision technology. At present, for thermal video input, natural color infrared imaging methods mainly follow the methods for static thermal images, perform independent color reconstruction on each frame or key frame, ignoring the time domain information, there are two shortcomings: 1) The time-domain features of object thermal radiation has not been used effectively, hindering the understanding of thermal imagery; 2) The time-domain smoothness of the reconstructed colors has not been constrained, resulting in the lack of between-frame consistency. This paper intends to solve the above problem by using the complementary spatio-temporal information and constraints provided by thermal image sequence. Taking the thermal image sequence as a whole in 3D spatio-temporal space, considering the feature extraction under spatio-temporal correlation, considering the global optimization of color reconstruction in spatio-temporal space, proposes the model construction methods of color reconstruction for the thermal sequence based on the three-dimensional spatio-temporal random field, and proposes the color night vision system quality evaluation methods toward the improvement of color reconstruction methods, with the achievement of a stable natural color thermal video as the ultimate goal, hence is able to bring significant benefits for both military and civil fields.

自然感彩色红外成像方法是红外彩色夜视技术的研究热点。目前针对热像视频输入，自然感彩色红外成像方法主要沿用对静态热像的处理方式，在每一帧或关键帧上独立进行色彩重建，这种方式忽视了时域信息，存在两点不足：1）景物热辐射的时域特征没有得到有效利用，影响了对热成像的理解；2）色彩的时域平滑性没有得到约束，影响了色彩重建结果在帧间的一致性。本项目拟利用热像序列提供的时空信息互补和约束关系解决上述问题，将热像序列作为一个三维时空中的有机整体，考虑景物热辐射特征的时空关联提取、考虑重建色彩的时空全局优化，提出基于时空三维随机场的热像序列色彩重建模型的构建方法，以及面向色彩重建方法改进的彩色夜视视频质量评价方法，以最终获得自然稳定的彩色红外视频为目标，对军事及民用领域都有重要意义。

瞄准自然感彩色红外成像技术中的景物热辐射时域特征没有得到有效利用及色彩时域平滑性没有得到约束两大问题，本项目基于自适应相关滤波响应融合、局部特征流、时空互信息提出了景物热辐射特征的时空关联提取的方法；基于图像/视频语义分割及类别色彩、稠密条件随机场模型提出了面向时空一致性的热像色彩重建模型构建方法；基于自动化场景解析模型提出了新的彩色夜视视频质量客观评价指标。所提出的方法形成从色彩重建模型构建到彩色夜视系统质量评价等一系列的基本理论方法，为实现自然感红外热像彩色夜视提供了关键理论支撑。