增强现实装配操作空间的深度感知理论与方法研究

Augmented reality (AR) technology integrates the virtual object to the real world with the help of 3D display technology, interactive technology, a variety of sensor technologies. It emphasizes the interaction and confusion of the virtual 3D models and the real world. AR technology can enhance a user's perception of the real world with information that is not actually part of the scene but is relevant to the user's present activity. To manipulate virtual 3D models by hands in augmented reality environment supports a free interactive manipulation for digital assembly. In this case, hands, virtual models and real objects component an augmented reality assembly space. How to percept the configuration relationship between the virtual models and real objects in the augmented reality assembly space is the key problem for capturing virtual models by hands. The science problem addressed in our proposal is intelligence perception of the interaction with virtual models by natural hands gesture. The objective of our proposal is to give a solution to confusion between virtual and real world, depth information modeling and interaction technologies with virtual models by free hands. This can support the improvement on digital assembly technology featured with natural interaction by hands. The studied issues addressed in our proposal included as, (1)approach to configuration modeling of real objects in augmented reality assembly space based on computer vision,(2)confusion between virtual models and real world in augmented reality assembly space,(3)computer vision based models for natural interaction by hands and models for assembly objective. In this proposal, we will formulate some models and make a solution for augmented reality assembly, and therefore we will hold creative patents in this area. To illustrate the proposed approach, we will develop a digital assembly system based on natural interaction by hands. This system will be used in assembly and maintenance for complex mechanical products.

增强现实是借助三维显示技术、交互技术、多种传感技术等把虚拟信息融合到真实环境的技术，强调虚拟对象与真实环境的融合与交互，并以此增强人对真实环境的认知能力。增强现实环境下，徒手操作虚拟物体为数字化装配提供了更自然的交互操作方式。手、虚拟零部件和真实环境构成了增强现实装配操作空间。在这样一种操作空间中，如何感知虚拟物体和真实物体的空间位置、相互关系是实现徒手操作虚拟物体的关键。本项目的科学问题是自然手势操作虚拟物体的智能感知，旨在突破虚实融合、深度信息感知，徒手操作虚拟物体的交互技术等方面的难题，促进具有自然交互特征的数字化装配技术的发展。项目具体研究内容包括：基于视觉的真实对象深度信息恢复方法；虚拟物体和实际场景的融合理论；基于计算机视觉的自然手势交互和装配语义模型。本项目预期在理论研究方面取得重要进展，获得发明专利，并开发出基于增强现实环境的徒手装配数字化系统，应用于复杂产品的装配和维护。

本项目着重研究如何通过计算机视觉对真实环境进行感知，获取其深度信息并进行恢复，从而为实现徒手交互、虚实融合提供理论依据与可行的技术方案。完成的主要研究内容包括：1）基于计算机视觉的徒手交互方法；2）研究了基于深度相机的手部深度信息提取算法及虚实融合方法；3）基于机器视觉的深度检测与环境感知技术以及4）双目视频的增强现实视频透视式头盔的研制。本项目已发表论文28篇，其中SCI检索9篇，EI检索7篇，核心期刊7篇。已撰写专著1部，已申请发明专利4项，已授权发明专利1项，已授权实用新型专利1项，培养博士生1人，硕士生10人。本项目的理论成果将为实现增强现实环境下的徒手数字化装配系统的实现奠定理论基础。直到2016年12月31日止，本项目的成果已初步实现转化，现已为某公司开发了AR辅助布线系统并在生产线上正式投入使用。