面向大数据计算的高吞吐量众核处理器关键技术研究

Throughput and scale-out ability are two key Performance indicators when big data computing executing on many-core processors, which influence the performance of system in two aspects particularly: firstly it needs the long latency to transmit data due to overlarge diameter of network-on-chip, which caused scale-out ability badly; secondly MapReduce spends too long time because the memory-on-chip architecture doesn't support merging of the middle <key, value> efficiently. Actually different network-on-chip topologies and memory-on-chip architectures can make throughout of big data computing vary by 50%, seriously affecting the performance of the system. The project aims at high throughout and strong scale-out ability. Related models are established which focuses on key scientific issues of structuring and deploying many-core micro architecture in big data computing environment. On this basis, the project studies design of low-diameter many-core network-on-chip, design of memory structure with strong scale-out ability, design of efficient combined optimization algorithm etc. deeply. These key technologies can realize the goal of strong scale-out ability and high throughout when executing big data computing on many-core processors, and resolve the problem that current many-core micro architecture does not match the characteristics of big data and its computing. Hence, the study is of great scientific significance.

吞吐量和横向扩展能力是众核处理器上执行大数据时的关键性能指标，其影响因素突出体现在两个方面：一是由于片上网络直径过大所引起的传输数据时延长，横向扩展性差；二是由于片内存储架构不能支持中间<key,value>对的快速合并所导致的MapReduce运算时间长。事实上，不同的片上网络拓扑和片内存储架构可使大数据计算的吞吐量相差达50%以上，严重影响了系统性能。 本课题以实现高吞吐量和强横向扩展为目标，围绕众核微体系结构在大数据计算环境下的构建与部署的关键科学问题，建立了相关模型。在此基础上，深入研究了低直径的众核片上网络设计、强横向扩展性的片上存储结构设计、同key的<key, value>对的高效合并优化算法设计等关键技术。这些关键技术的突破能够实现众核处理器上执行大数据计算的高吞吐量和强横向扩展目标，解决现有的众核微体系结构同大数据计算特征之间的不匹配问题，具有重要的科学意义。

为实现高吞吐量和强横向扩展的目标，本课题设计了高通量众核微体系结构HTMA（High Throughput Many-core Architecture）。主要包括：建立了面向大数据计算的众核片内数据传输模型，提出了低直径的众核片上网络V-Mesh结构，强横向扩展性的片上存储结构以及同key的<key, value>对的高效合并优化算法设计等关键技术。HTMA围绕众核微体系结构在大数据计算环境下的构建与部署的关键科学问题，实现了众核处理器上执行大数据计算的高吞吐量和强横向扩展目标，解决了现有的众核微体系结构同大数据计算特征之间的不匹配问题。本课题的研究成果可以应用在机器人行业的类脑芯片、虚拟现实仿真以及计算机视觉等领域，因而具有重要的科学意义。