航拍视频全景拼接、前景提取及压缩技术研究

In last years, the aerial photography has become more and more popular and been widely applied to many fields, such as military survey, target surveillance and city planning, which is capable to perform tasks in high altitudes and in dangerous situations mainly thanks to its flexibility. But the flexibility also limits the load of carrier. Thus, it is critical for the device to effectively process the captured data and efficiently compress the data and transfer it to receivers. This proposal aims to provide a novel solution for intelligently processing and compressing aerial videos based on the characteristics of the aerial videos, i.e. the large scale, high resolution and potential, which mainly contains three parts: 1) pixel-wise panorama stitching, 2) accurate foreground detection and 3) effective video compression. More specifically, we propose to simultaneously align frames with overlapping in a pixel-wise manner and stitch the panorama as well as construct the background dictionary. For detecting foregrounds, the probabilistic model of foreground could be modeled by exploiting the spatial-temporal smoothness and the uniform distribution of color/intensity of foreground. To save the storage space, this proposal attempts to reconstruct the background of each frame via sparse coding with respect to the background dictionary, which is the origin of compression. As for the detected foreground, we plan to shrink the searching space of moving vector by enforcing that only on the part masked as the foreground, which would significantly reduce the residual between the matched macro-blocks and improve the quality of the compressed video.

近年来，航拍技术在诸多领域，如军事勘察、目标监控及城市规划等，得以重用。遥感航拍在高空域和危险地区的作业能力得益于它的灵便性，但同时也导致其载重限量。自然地，其硬件设备对所采集的数据快速分析及高效压缩存储的能力显得尤为关键。本课题拟从航拍视频的特点，即大比例尺、高清晰度及高现势，为切入点，为航拍视频的智能处理和高质量压缩提供新思路。本课题拟从三个方面深入展开研究，包括1）精密的全景拼接：对视频帧像素级别的对齐，进而获取视野更为广阔的全景图得以宏观掌握目标区域情况，同时获取背景字典；2）精确的前景提取：为了精确的从视频中分割前景物体，利用前景物体的时空平滑性和颜色/灰度值得分布特性从概率的角度对其进行建模；3）高质量的视频压缩：凭借全景图定位每一帧视频的相对位置，利用背景重建消除大量的冗余信息。对于前景，拟通过缩小压缩时宏块运动向量的搜索空间来提高搜索效率和宏块匹配准确度。

航拍技术在很多领域，例如军事勘察、目标监控及城市规划等，已经展现出巨大的潜力。本项目从航拍特点切入，主要展开了以下三个方面研究：1）用以支撑全景拼接的高性能低秩矩阵恢复技术；2）用以精确分离物体的前景分割、图像配准、低质量数据增强技术；以及3）结合航拍视频特点的高质量视频压缩技术。本项目提出一系列的技术方案突破问题瓶颈：a）通过改变优化模型和引入在线方式处理视频帧加速低秩矩阵恢复，并通过提出外点数据提升恢复精确度，b）利用背景强相关性以及前景时空特性提高前景分割精度，和c）减少对图像变换的模型依赖，仅利用局部结构信息对图像序列快速配准；并尝试通过对视频空间冗余消除和减少前景运动向量搜索空间提升压缩性能。部分成果已发表于领域顶级会议和期刊。