基于点云深度学习的光机组件三维位姿估计与跟踪

Large laser facility is highly significant for national security and energy. Estimating and tracking 6-degree-of-freedom (6DOF) pose of opto-mechanical assemblies is a key enabling technology for the integrated installation of the large laser facility, thus it has important significance in applications and research. There are some difficulties in estimating and tracking the 6DOF pose of opto-mechanical assemblies, such as high precision and cleanliness requirements, the background clutter and the opto-mechanical assembly’s texture-less appearance. Against the above mentioned difficulties, the project utilizes active 3D camera to acquire the point cloud data with a non-contact manner and with high precision, and introduces generative adversarial neural network and deep feature learning that directly consumes point clouds to generate candidate regions that likely compactly contain the desired opto-mechanical assemblies quickly and to learn good intrinsic point cloud features automatically. This project expands research in the following four aspects: (1) 3D object proposals generation using a point cloud generative adversarial network; (2) Point cloud feature learning for estimating and tracking the 6DOF pose of opto-mechanical assemblies; (3) opto-mechanical assembly’s 6DOF pose estimation via 3D object proposals generation and point cloud feature learning. (4) Tracking the 6DOF pose of opto-mechanical assemblies via semantic segmentation and Iterative Closest Point algorithm, with the former using deep feature learning on point cloud. Finally, we establish a 6DOF pose estimation and tracking prototype system for the opto-mechanical assemblies of the large laser facility. Then apply and verify the prototype system in the maintenance of the newly built large-scale laser facility, and provide technical support for the construction of our country’s future ignition facility.

大型激光装置对国防安全、能源等意义重大，光机组件三维位姿估计与跟踪是实现其集成安装的关键技术，具有重要的应用意义和研究意义。课题针对光机组件三维位姿估计与跟踪中的难点和问题，如精度和洁净度要求高、背景杂乱、表面弱纹理等，使用三维视觉传感器非接触式地获取高精度点云数据，引入点云生成式对抗神经网络和点云深度学习，快速生成候选区域和自动学习良好的点云特征，并从四个方面展开研究：1.基于点云生成式对抗神经网络的光机组件三维似物性采样;2.面向光机组件三维位姿估计与跟踪的点云特征学习;3.三维似物性采样与点云特征学习使能的光机组件三维位姿估计；4.基于点云特征学习语义分割和最近点迭代的光机组件三维位姿跟踪。最终形成有效的光机组件三维位姿估计和跟踪方法，并建立相应的位姿测量原型系统，在我国最近建成的大型激光装置维护作业中进行验证，并为我国即将建设的下一代大激光装置提供技术支撑。

大型激光装置对国防安全、能源等意义重大，光机组件三维位姿估计与跟踪是实现其集成.安装的关键技术，具有重要的应用意义和研究意义。课题针对光机组件三维位姿估计与跟踪中的难点和问题，如精度和洁净度要求高、背景杂乱、表面弱纹理等，使用三维视觉传感器非接触式地获取高精度点云数据，引入点云深度学习，对光机组件三维位姿估计与跟踪进行了深入研究。取得的主要进展包括: ①对点云深度学习方法进行了综述；②提出了系列基于深度学习和注意机制的目标检测及分割方法；③提出了基于图网络的点云特征学习方法，并在三维目标检测任务上验证其有效性；④提出了系列基于点云深度学习的物体抓取位姿估计方法；⑤基于上述模型的光机组件三维位姿估计方法；⑥ 建立一个适用于大型激光装置光机组件安装的三维位姿测量原型系统，在我国最近建成的大型激光装置是的多类光机组件上进行了评估实验和验证。项目共发表论文22篇，申请专利3项，培养博士后1人，博士生1人，硕士生3人。项目研究形成的光机组件三维位姿估计和跟踪方法，对于提高大型激光驱动器光机组件装配自动化具有重要意义，并可推广应用至服务机器人及工业机器人的灵巧操作领域。