信息物理融合系统中多模态数据的融合机制研究

The complexity and heterogeneity of cyber-physical systems (CPS) pose a new challenge to observe and cognize the physical world accurately and comprehensively. CPS obtains the information of physical world by analyzing and fusing the multi-modal data generated by heterogeneous equipments. However, the research on integrating multi-modal data is still at the initial stage, which has not been solved effectively for long. To solve this problem, this project aims to conduct a thorough study on the key techniques and theories of integrating and computing multi-modal data. By exploiting the correlations among multi-modal data, this project will design a suite of high efficiency, real-time and light-weighted algorithms for cognizing the physical world, which include: 1) the multi-modal data associated queries processing algorithms, aiming at efficiently obtaining information of the physical world represented by multi-modal data and providing the basis for cognizing the physical world; 2) multi-modal data compression and storage techniques, with the target of diminishing the scale of the data and conserving storage resources by exploiting the correlations of multi-modal data; 3) multi-modal data conjoint analysis and mining mechanisms, with the objective of building the deep connections of multi-modal data and providing a strong support to understand the physical world. On the basis of the research results above, this project will develop a system prototype to confirm the effectiveness and efficiency of the proposed techniques.

信息物理融合系统的异构性和复杂性为正确全面地观察和理解物理世界提出了新的挑战。系统通过融合异构设备产生的多模态监测数据来获得物理世界的信息。然而，针对多模态数据融合问题的研究还处于起步阶段，至今未得到有效的解决。为此，本项目针对面向物理世界认知的多模态数据融合与计算的关键技术和理论进行深入研究，利用多模态数据之间的相关性和协同互补性，设计一套面向物理世界认知的高效率、实时性、轻量级的多模态数据融合算法，包括：1)多模态数据关联查询算法，目标是高效的获取多模态数据描述的物理世界的信息，为认知物理世界奠定基础；2)多模态数据的联合压缩存储技术，目的是利用多模态数据之间的相关性来降低数据的规模；3)多模态数据的联合分析与深层挖掘机制，目标是进一步发现多模态数据之间的关联，为正确的理解物理世界提供保障。基于上述的阶段性研究成果，本项目还将研制通用的原型系统，验证基础理论研究成果的有效性和可行性。

信息物理融合系统是一个综合计算、网络和物理环境的多维复杂系统，通过将计算技术、通信技术和控制技术的紧密融合与深度协作，实现信息世界与物理世界的紧密融合。信息物理融合系统中的多模态数据对物理对象进行多维属性、多侧面、多视角的度量，这些多模态数据互相补充和完善，共同提供物理世界状态的有效依据。针对多模态数据带来的诸多挑战，项目组充分利用多模态数据之间的相关性和协同互补性，提出了基于多模态数据的面向物理世界认知增强的深度分析模型和处理算法，包括低质多模态数据的深度融合和查询处理方法、多模态数据的深层联合分析方法以及多模态数据深度融合中的高效计算方法等，并通过仿真实验和理论分析的手段评价了提出方法的效果和性能。本课题执行期间取得了良好的进展，多个研究成果在国内外高水平的学术会议和期刊发表，培养多名研究生。本项目通过深度整合多模态数据，进而更全面、精确地观察和理解物理世界，进一步加深了信息物理融合系统关于物理世界认知的理论与应用。