基于光场的高动态范围超分辨率成像研究

High Dynamic Range (HDR) and Super Resolution (SR) are the two basic requirements for high quality digital imaging, and both are hot and difficult issues in multimedia and Computer Vision research area, while Light Field (LF) imaging is the cutting edge topic during international academic. Combining these matters into one task generates new challenges. This project explores the basic theory on sampling and reconstruction of Light Field, analyzes fundamental factors affecting Light Field imaging, and investigates theory and technology of Light Field imaging with High Dynamic Range and Super Resolution. The main research contents include the followings. (1) Firstly, exploring internal relations between Light Field data, including light's radiance here, and imaging's dynamic range and resolution; studying effective sampling method obtaining both Light Field and radiance data simultaneously with sparse theory. (2) Then, analyzing properties and influence factors of Light Field imaging to construct image formation model suitable for High Dynamic Range and Super Resolution processing, while seeking relationship between both scene's depth and radiance information and both High Dynamic Range and Super Resolution process, and using Manifold Learning (ML) theory to recover scene's elaborate depth map and radiance map. (3) Finally, attributing the main problem as reconstruction of 4-dimension Light Field, which is decomposed into two 2 dimensions sub- Light Fields: Image Plane and Lens Plane, and their High Dynamic Range and Super Resolution information are reconstructed respectively. Image Plane is reconstructed from scene's depth map and radiance map using priors under Bayesian framework, while Lens Plane is reconstructed based on sparse dictionary created with Compressive Sensing (CS) theory. 4 dimensions' Light Field is reconstructed by the synthesis of both sub- Light Fields. Based on reconstructed Light Field, High Dynamic Range and Super Resolution image can be formulated.

高动态范围与超分辨率成像是多媒体与计算机视觉领域的热点与难点问题，光场成像则是目前国际上研究的前沿课题，将它们进行综合研究带来新的挑战。本项目探索光场采样与重建的基本原理，分析影响成像的根本因素，研究高动态范围和超分辨率的光场成像的理论与技术。（1）首先，探索包含照度在内的光场数据与成像动态范围和分辨率之间的内在联系，应用稀疏理论研究对场景的光场和照度数据同时采样的有效方法。（2）然后，分析光场的成像特性及影响因素，研究适合高动态范围和超分辨率处理的光场成像模型；探索场景中深度与照度信息与光场超分辨率、高动态范围处理之间的联系，应用流形学习理论解析出场景的连续深度图与照度图。（3）最后，将研究问题归为四维光场的重建，并分解为二维的像平面和镜平面的重建。对像平面应用贝叶斯框架下的先验假设从场景的深度图与照度图实现重建；对镜平面应用压缩感知理论建立稀疏字典实现重建；综合二者得到四维光场的重建。

光场理论及其成像技术与应用是当前国际上研究的前沿课题, 本项目探索光场采样与重建的基本原理，分析影响成像的根本因素，提高光场成像在高动态范围与分辨率上的提升。针对所研究的内容及相关领域，搭建、改进了两个相应的光场数据采集系统。一个系统通过设计可编程的光照，不仅能够完成对静态场景不同的光照图像采集，而且通过视点改变光照角度，刻画出同一物点对于不同光线耦合的作用，实现高动态范围的光场采集。另一个系统提供全景多光照动态光场平台，包括320个光源和40台摄像机，将密集全景光照阵列与密集全景摄像机结合起来，获得更具完备性的多视点多光照数据，并设计了基于分布式的多光照、多摄像的控制系统，获得了灵活可变的数据采集方式。实现了一个超高像素的分辨率视频采集装置，解决了视场角和分辨率的矛盾，具有视场角大、分辨率高的优点。同时，在光谱数据的特征挖掘、场景深度分析与匹配、基于流行学习的场景分类、基于压缩感知的成像处理、基于稀疏表示的场景分析等方面取得进展。发表学术论文6篇，其中SCI期刊2篇，另有2篇论文被国际会议录用。获得国家发明专利授权9项。人才培养方面，项目组负责人刘宇驰及成员黄志良获得副教授职称。项目组成员付长军、徐枫、邓岳、李冠楠获得博士学位。本项目的研究思路与研究成果将促进对光场与成像机理的深入探索，并提高光场理论在实践领域应用水平。