复杂室外环境下的大视场、大范围场景感知研究

Vision sensor is an important means for scene capturing of mobile robots. However, in unstructured and uncontrolled outdoor enviroments with large amount of uncertainties and complexities, traditional vision-based approaches suffer from the limitations imposed by narrow field-of-view (FOV), short-range detection and low sensitivity to scene changes, and thus cannot meet the application demands of outdoor robots. Vision-based perception in complex outdoors is still a key and urgent problem to be solved. To overcome these limitations, a wide-FOV and long-range perception scheme is proposed in this proposal, where both the vision theory and acquisition manner will be investigated together. In specific, the omnidirectional visual acquisition and dense scene reconstruction are addressed to achieve wide-FOV acquisition and analysis. A deep network is built based on transfer learning and semi-supervised fashion to achieve adaptive feature extraction. A near-to-far self-supervised architecture is designed to achieve long-range perception. Fundamental theories and technical hierarchies for wide-FOV and long-range perception in complex outdoor scenes will be established in this project. Feasibility and effectiveness of the whole system will be verified with quadruped and wheeled robots. The project outcomes can provide sound theoretical foundations and key technical supports to significantly improve the autonomous capabilities of mobile robots in complex environments.

视觉传感器是移动机器人进行信息采集的重要手段。但室外环境景物多变、非结构化及无规律性明显,传统视觉感知由于采集视场小（窄视）、感知距离近（近视）、环境变化适应性差（弱视），无法满足室外机器人的应用需求，基于视觉的复杂室外环境场景感知已成为机器人领域一个亟需解决的关键问题。本项目打破传统方法的局限，另辟蹊径，将视觉采集模式改进与算法理论研究相结合，提出大视场、大范围感知体系。通过构建全向视觉采集系统及基于全向视觉的场景密集重建进行大视场信息获取和分析，通过深度网络基础上的迁移半监督学习进行场景特征的自适应提取，通过由近及远的自监督在线学习进行远距离大范围场景感知。本项目旨在建立一套针对复杂室外场景的大视场、大范围的机器人自主感知的基本理论和方法体系，并通过四足、轮式机器人移动平台验证相关理论和技术的可行性和有效性，为全面提高移动机器人在复杂环境下的自主性提供重要理论基础和关键技术支撑。

本项目组成员通力合作，依照项目计划书顺利完成了各项内容，并通过本项目的资助取得了一系列成果。具体地，本项目建立了多相机协同视觉采集系统并通过基于全向视觉的场景感知实现了大视场信息获取与分析，通过构建端到端学习的深度网络模型提取具有任务适应性的场景语义特征，通过由近及远的自监督学习进行远距离大范围场景感知，最终初步建立了针对复杂场景的大视场、大范围场景感知方法体系，并搭建了基于多相机协同的机器人感知验证平台，借助该移动平台验证了大视场、大范围场景感知技术的可行性和有效性，为全面提高移动机器人在复杂环境下的自主性提供了理论和技术支撑。相关成果在国内外知名学术期刊和会议发表SCI论文23篇，EI论文11篇，其中TPAMI、TIP、TNNLS、TCYB、TCSVT、TMM等IEEE汇刊14篇，ICCV、CVPR、IJCAI、AAAI、IROS等人工智能与机器人领域顶级会议论文8篇，另有8篇论文正在审稿过程中；相关论文得到国内外同行学者的引用和正面评价，其中2篇入选ESI高被引论文，1篇入选ESI热点论文；获发明专利授权4项，另有2项发明专利进入实质审查阶段；获国家自然科学基金联合基金重点项目资助1项、重大项目课题资助1项；获CVPR2016大规模场景理解挑战赛冠军、CVPR2018大规模行为识别挑战赛季军。