分层三维重建学习

Recent years, machine learning has revolutionized the visual object vision field, for example, deep networks and deep leaning have achieved comparable results to human vision system on visual object segmentation, identification and categorization. However, in spatial vision, such as 3D scene reconstruction, object pose estimation, the learning based methods are much less performing compared with the currently widely used geometry-based methods. One of the key underlying reasons is that the current learning based methods generally lack an embedded outlier-remover of point correspondences, such as RANSAC, hence they are invariably sensitive to correspondence outliers, whereas such an outlier-removal module can be conveniently integrated into the geometry-based methods. The main objectives of this proposal are two-fold: (1): Under supervised learning, how to robustly learn projective reconstruction, affine reconstruction and metric reconstruction from putative multi-view point correspondences, in particular, how to embed an outlier-remover in the current network architectures to enhance the robustness to mismatches and to improve the completeness of the reconstructed scene. (2): Under unsupervised learning, investigate whether learning stratified 3D reconstruction is possible from pure image point correspondences, in particular, whether it is possible to upgrade the projective reconstruction to affine reconstruction by learning in such cases. To our knowledge, there has been no related report in the literature on the above two issues up to now. In addition, our proposed methodology, implementation techniques and the expected outcomes could be of reference for both machine learning and geometry people in computer vision.

近年来深度学习在物体视觉方面，如物体分割与识别等，取得了可与人类视觉相媲美的性能。但在空间视觉方面，如三维重建，物体定位等，深度学习方法的性能却远不如传统的基于几何的方法。究其原因，申请人觉得主要是由于传统基于几何的方法中，可以很方便地集成诸如RANSAC等鲁棒模块从而剔除多幅图像之间不可避免的匹配外点，而基于深度学习的途径，目前却很难对图像匹配外点进行剔除。 本项目的主要目标包括二方面：（1）：如何通过监督学习的途径达到场景的射影重建、仿射重建和度量重建，重点探索如何在深度学习的框架下引入鲁棒剔除匹配外点的机制，以提高三维重建学习的鲁棒性和完整性。（2）：探索在无监督下仅仅基于图像特征的对应关系，能否将射影重建通过学习途径提升到仿射重建和度量重建。本项目的研究内容目前文献中还没有见到过类似报道，项目的研究方法和思路对计算机视觉界从事几何研究和机器学习的其他人员均会有一定的参考价值。

分层三维重建是计算机视觉中一种众所周知的三维重建方法，是包括如街景重建、倾斜摄影测量、智慧3D等的关键技术。本项目重点研究了以下二方面的内容：（1）研究基于学习的分层三维重建方法，该方法使用跨图像的匹配点对恢复场景的三维射影结构，以及如何基于学习的重建方法中显式或隐式地集成外点检测器是学习重建三维场景结构。我们的结果表明，从噪声图像点对应中学习分层三维重建确实是可能的，并且学习的重建结果令人满意。（2）项目执行期间，除理论算法外，我们还研发了大场景三维重建系统。该系统既可以重建地面图像或无人机图像，也可以将二种不同重建的结果进行融合。