面向新型异构众核系统的多设备协同并行计算关键技术研究

Heterogeneous many-core systems consisting of multiple multi-core CPUs and many-core accelerators have been widely used in high-performance computing due to their advantages of high performance, low power consumption and low cost, and more and more parallel applications are developed in such a heterogeneous system. However, the lack of an easy-to-use heterogeneous cooperative parallel programming model and the complicated system architecture have seriously restricted the development efficiency and perform efficiency of the parallel applications based on heterogeneous platform. In this project, we firstly propose a directive-based heterogeneous cooperative parallel programming framework, in order to reduce the difficulty of heterogeneous cooperative parallel programming and the programming burden of the programmers and efficiently support the multi-device cooperative parallel computing of the data-parallel applications based on heterogeneous platform. Secondly, we focus on utilizing multiple compute devices to reasonably, efficiently and cooperatively execute a data-parallel application on a heterogeneous many-core system, a preemption-based dynamic and elastic task scheduling strategy is proposed. Finally, considering that the inter-device communication can easily become a performance bottleneck of heterogeneous cooperative parallel computing, we propose a three-way overlap communication optimization method based on software pipelining in order to effectively hide the communication overhead between devices. This research will provide support of the easy-to-use programming framework, efficient task scheduling and communication optimization for the multi-device cooperative parallel computing, and it also will provide a feasible way to improve the usability and computational performance of the heterogeneous many-core system.

由多核CPU和众核加速器构成的新型异构众核系统以其高性能、低能耗和低成本等优势在高性能计算领域得到迅猛发展，基于异构众核系统开发的并行应用也日益增多。然而，简单易用的异构协同并行编程模型的缺乏以及异构众核系统复杂的体系结构严重制约了基于异构平台的并行应用的开发效率和执行效率。项目首先研究基于指导语句的异构协同并行编程框架，以降低异构协同并行编程难度、减轻编程负担并有效支持基于异构平台的数据级并行应用的多设备协同并行计算；然后，研究如何在异构平台中利用多个计算设备合理高效地协同执行数据级并行应用，提出抢占式动态弹性任务调度策略；最后，鉴于设备间通信易成为异构协同并行计算的性能瓶颈，提出基于软件流水线三路重叠式通信优化方法，以有效隐藏设备间通信开销。本研究将为面向异构众核系统的多设备协同并行计算提供简易编程框架、高效任务调度和通信优化支撑，为提高异构众核系统的易用性和计算性能提供一种可行方法。

由于异构众核系统中不同类型的计算设备具有不同的体系结构、计算能力、存储能力和通信能力，这严重制约了基于异构平台的并行应用的开发效率和执行效率。为此，项目首先构建了适合于数据级并行应用的基于指导语句的异构协同并行编程框架。通过扩展最新OpenMP 5.0的导语和子句给编程人员提供了一种更为简单的方式来充分利用异构众核系统中所有可用的计算设备来协同执行数据级并行应用，借助本项目开发的源到源的编译器和运行时系统能将计算任务和数据自动合理分配到参与协同并行计算的各设备中。然后，项目重点研究了高效支持异构协同并行计算的异步抢占式动态弹性任务调度策略，通过在数据级并行应用程序的运行过程中灵活动态地调整不同设备的工作负载，显著减少了频繁的设备间数据传输和设备间同步所带来的开销、最大化了各计算设备的利用并保持较好的设备间负载均衡。最后，项目探讨了能有效隐藏设备间通信开销的基于软件流水线三路重叠式通信优化方法，在确保PCI-E总线带宽和加速器端计算资源得到充分利用的前提下，最大程度重叠了加速器端计算内核执行与主机−加速器间数据传输，并将三路重叠式通信优化与异步抢占式动态弹性任务调度策略融合在一起，实现了能有效隐藏设备间通信开销的任务调度。本研究能在一定程度上促进面向异构众核系统的多设备协同并行计算的理论探索，对于提高面向异构众核系统的数据级并行应用的开发效率与总体计算性能具有重要理论与应用意义。