面向视频卫星的多分辨率字典学习压缩视频超分辨率重建

Newly emerging video satellites can capture motion videos and thus greatly enrich the dynamic observation capacity of satellite remote sensing systems. However, due to the performance of imaging devices as well as the limited transmission capacity of celestial channels, the clarity of dynamic video achieved by video satellites is of great insufficiency. Existing multi-frame fusion and reconstruction based video super-resolution techniques are difficult to obtain sufficient high-frequency information from the over compressed video. To this end, this proposal focuses on compressed video super-resolution based on multi-resolution dictionary learning. In the light of the identically distributed granularity law between variable block sizes in compressed streams and homogeneous landscape areas with similar textures and appearance structures, this application proposes the multi-resolution dictionary structure driven by multiple encoding schemes to promote the representation accuracy of the dictionary on heterogeneous regions. Meanwhile, according to the rigid nature of the movements of ground targets resulted from ultra long-distance imaging by video satellites, the proposal introduces temporal constraints of previously reconstructed high-resolution frames, and formulates joint temporal-spatial sparse representation accounting for sparse consistency of adjacent frames to enhance temporal smoothness of motion objects. Besides, this proposal introduces deep learning to train the relation mapping between low- and high-resolution coefficients in sparse domain, and then uses the corrected accuracy ones to promote the fidelity of super-resolved reconstructed videos. The spatial resolution of motion imagery of video satellite is expected to be doubled so that the requirement of high-precision interpretation on the motion object imagery could be satisfied in dynamic earth observation.

近年来兴起的视频卫星极大地丰富了卫星遥感系统的动态观测能力，但受限于成像器件性能和星地信道传输能力，视频卫星动态视频的清晰度严重不足。现有的基于多帧融合重建的视频超分辨率技术难以从过度压缩的视频中获得足够的高频重建信息，为此，本申请研究基于多分辨率字典学习的压缩视频超分辨率重建技术。首先，从压缩码流中可变尺寸编码块与具有相似纹理和表观结构的地貌同质区粒度同分布的特性出发，提出多编码模式驱动的多分辨率字典学习模式，提升字典对地物异质区域的表达精度；其次，根据视频卫星超长距成像带来的地表目标的刚体运动性质，建立相邻帧字典投影系数一致性约束的时空联合稀疏表达模型，增强运动目标的时域平滑性；最后，将深度学习思想引入到高低分辨率系数映射关系的发现中，利用映射后的精确系数提高重建视频的保真度。预计将视频卫星动态视频的空间分辨率提高1倍，满足对地观测中动态目标影像高精度解译的需求。

视频卫星极大地丰富了卫星遥感系统的动态观测能力，但受限于成像环境和星地信道传输能力，视频卫星动态视频的清晰度严重不足。现有视频超分辨率技术难以从过度压缩的视频中获得足够的高频重建信息，为此，本项目从如下几方面开展研究：.（1）针对视频卫星影像中地物类型复杂而多样化的特点，提出超密集连接学习模式，突破了图像内容复杂性自适应的表达和灵活特征提取的技术瓶颈。.（2）针对超高视距成像及过度压缩引起的地面目标边界淡、纹理弱的问题，提出边缘增强的生成对抗学习网络，提升了小尺度、弱纹理地面目标的边缘轮廓重建精度。.（3）针对重建视频帧存在的闪烁、抖动、拖影等时域不一致现象，提出非局部时空相关性建模及时域一致的重建方法，突破了动态视频空域逼真、时域稳定重建的技术瓶颈。.（4）针对高低维空间映射关系固有的偏差问题，提出耦合稀疏自编码器学习低分辨率图像和目标高分辨率图像之间的空间映射关系，实现高低维隐含关系的准确映射。.以上述理论成果为主，项目组在国内外权威期刊上发表SCI论文20篇，顶级国际会议论文6篇；并对其中部分关键技术申请了知识产权，申报发明专利11件，已授权5件，成功转让2件；获得IEEE AIM 2019 Challenge on Video Extreme Super-Resolution亚军、NTIRE 2019 Challenge on Video Super-Resolution第三名、2020年中国人工智能产业发展联盟“旷视人工智能开源大赛”第三名。.成果应用到珠海横琴环岛边海防电子围网、智慧城市视频大数据时空分析平台、平安城市视频可辨识度增强解析系统等重要系统的研制中，用于提升卫星影像和地面监控视频的清晰度，为精确监测城市地理信息要素及监控社会治安目标的下游任务提供支撑。项目组参与申报的“跨域多维电子围网关键技术及应用”获得2019年广东省科技进步一等奖、“城市视频大数据时空智能分析关键技术研究及应用”获得2019年中国地理信息产业协会地理信息科技进步一等奖、“视频可辨识度增强解析技术与应用”获得2020年公安部科学技术一等奖。