基于沉浸式的航空维修遥操作技术研究

The aero-engine is a electromechanical system with complicated structure and highly intensive technology, and its application and maintenance needs proficient technicians. Immersive robot teleoperation technology is introduced into aeronautical maintenance, so as to improve the efficiency of maintenance staff. Combined with large-scale virtual reality equipment (CAVE) stereoscopic display, augmented reality and optical tracking technology, scene understanding and depth information are used to achieve dynamic reconstruction of mixed three-dimensional scene. With the optical tracking device, gaze tracking devices and virtual hand model, multi-channel teleoperation interaction method based on body posture tracking, gestures and sight line is studied. By accurately tracking the operator, overlapping and mapping virtual enhanced information into the video of real scenes, and integrating the operator into the work environment and object of remote device, the previous precise location of human-computer interaction is expanded the entire environment, thus improving the operability of aviation maintenance system and enhancing the telepresence of teleoperation system. With the theories and research methods above, the project will establish the theoretical and technical foundation about how to construct an immersive teleoperation system with good operability and telepresence.

民用航空器是一种大型的结构复杂、技术高度密集的机电系统，它的使用和维修对机务维修人员的水平要求较高，本课题拟将沉浸式遥操作机器人技术引入到航空器维修中，以提高维修人员的工作效率。结合大型虚拟现实设备（CAVE）立体显示、增强现实和光学跟踪技术，利用场景理解和深度信息实现三维混合场景动态重建；将光学跟踪设备、视线跟踪设备和虚拟手模型相结合，研究基于人体姿态跟踪、手势、视线相结合的多通道遥操作交互方法。通过操作人员的精确跟踪、定位，将虚拟增强信息叠映于真实场景视频流之中，操作者融合于远程设备工作的环境与对象中，使遥操作人机交互从精确的位置扩展到整个环境，改善了航空维修系统的可操作性，增强了遥操作系统的临场感。本课题通过以上基础理论和方法研究为研制具有较强的临场感、可操作性的沉浸式遥操作系统奠定理论和技术基础。

本项目紧紧围绕沉浸式遥操作环境中三维混合场景动态建模和多通道人机交互两大关键问题进行研究。课题组将深度摄像机和立体（双目）摄像机相结合，基于深度信息和场景理解实现了混合场景动态建模，增强了沉浸式遥操作系统的临场感；设计了沉浸式环境中单目平面视线跟踪和双目空间三维视线追踪两套系统，结合人体姿态、手势初步解决了沉浸式环境中的人机自然交互问题，建立交互模型并提高了算法计算效率；为了验证所提出理论模型的实际效果，课题组在工业模型读取和解析、工作流程获取与编辑、三维场景采集与感知、人机自然交互与控制、增强信息融合与显示五个方面进行了沉浸式遥操作原型系统核心模块开发工作，完成了沉浸式遥操作原型系统的硬件和软件平台构建。项目资助在国际会议和国内期刊发表论文10余篇，获得授权发明专利1项，实用新型专利3项。