面向高维数据空间的张量异构并行计算方法研究与探索

More and more high dimensional data have been generated in in the Internet and the Internet of things. The matrix is difficult to describe the features of its high dimensional data, but tensor is one of the most natural ways to describe a high dimensional data space. The high dimensional data is usually characterized by large scale and super sparsity, and tensor algorithm is quite effective in dealing with such complex high dimensional problems. In particular, tensor decomposition is an important means to explore the intrinsic data relationship in high dimensional data space. However, with the continuous development of large data applications, the computational complexity of tensor decomposition has been approximated to the E-level computing, and large scale parallel computing is needed to meet its computing requirement. This topic will study the scalable heterogeneous parallel algorithm of large-scale high-dimensional tensor operation and its performance optimization method based on large-scale heterogeneous multi-core / multi core parallel computer system to meet the computational requirement of high-dimensional space machine learning. The study try to explore the theory and break through technical problems for dimensionality reduction algorithm,task partitioning, and multi-level parallel of parallel algorithm for tensor operation, and form a set of efficient heterogeneous parallel subprograms for tensor operations on heterogeneous parallel computer systems. The research of this topic is of great importance not only for efficient parallel processing of tensor numerical operation, but also for the development of machine learning common algorithm library on large-scale heterogeneous parallel computer systems. Moreover, the research results also contribute to solving challenging issue of machine learning in scientific and engineering fields, such as high-dimensional community discovery, web search, Image and video processing, and biological medicine, etc.

在互联网和物联网应用中产生了越来越多的高维数据，张量是描述高维数据空间最自然的方法。高维数据通常具有大规模和稀疏等特征，而张量算法应对复杂高维问题相当有效，尤其张量分解是探寻高维数据空间内在数据关系的重要手段。随着大数据应用的不断深入，张量分解的计算量已向E级计算逼近，需要大规模并行计算才能满足其计算需求。本课题将基于大规模异构并行计算机系统，针对高维空间机器学习的计算需求，研究大规模高维张量运算的可扩展异构并行算法及其性能优化方法，力图探索和突破面向异构并行计算机的张量运算并行算法中所涉及的降维压缩、任务划分和算法多级并行等理论和技术难题，形成一套高效的张量运算异构并行算法库。本课题的研究不仅对张量运算的高效能并行处理和异构并行计算机上的机器学习共性算法库研制具有重要意义。而且，项目研究成果对解决与张量相关的大数据和人工智能领域的挑战问题也具有相当的促进作用。

项目针对大数据、推荐系统、视频图像处理领域产生的大规模高维张量数据，由于张量数据运算的计算量随着维度的增加而呈指数级增长，而目前我国国产自主的高性能计算系统大多数异构计算系统，因此本项目主要在异构并行计算平台和国产计算平台上对张量运算算法和性能优化及其张量库的应用展开研究，主要分为四个项目内容：大规模稀疏张量的压缩存储方法研究、并行矩阵运算的性能优化研究、张量运算的并行算法研究和设计和张量在领域应用中的并行算法设计和性能优化技术研究，提出了描述大规模稀疏张量非零元素分布的概率密度函数，解决了不同压缩方法下访存效率预测难题。并针对异构处理器结构研究和设计了异构并行计算方法，针对张量的基础矩阵并行计算算法，在国产自主计算平台上性能比目前主流开源的软件性能均有明显提升。基于自研的基础矩阵运算算法，针对CPU+GPU异构计算环境研究和设计异构并行的张量运算并行算法，并在天河三号、海光服务器华为鲲鹏服务器上实现了完全自主的张量运算库，并已经进行部署应用，针对自研的张量运算库开展了在交通图像处理、超声图像诊断、推荐系统等领域的应用研究，获得4项发明专利授权，进行成果转换。本项目针对张量运算的研究为在高维数据的降维、特征提取和补全提供高性能的算法支持，为在国产异构高性能计算机平台上开展大数据智能和AI模型训练、数值模拟仿真提供基础的数值运算的软件库支撑。本项目的研究成果发表学术论文16篇，其中SCI/EI检索13篇，CCF推荐的A类期刊和会议2篇，CCF推荐的B类期刊2篇。中文核心期刊3篇。获得发明专利授权4项，获得湖南省自然科学一等奖和二等奖各1项，2019年产学研合作创新成果奖一等奖和2022年中国计算机学会科技成果特等奖。