面向3D XPoint的搜索引擎索引和缓存技术研究

In the Internet age, search engine is the most important Internet portal. After new storage devices, such as 3D XPoint etc., are used in search engine servers, the logs of datacenter showed a significant rise to the server CPU load, even may affect the operation of the server. The reason of this problem is that the high I/O performance of the new storage devices make the bottleneck of system migrate from storage I/O to CPU computing. The goal of the research is to redesign the search engine cache mechanism and index structure according to the features of new storage devices such as 3D XPoint, to decrease CPU load. Specifically, the research content includes: (1) the idea of cache mechanism in search engine changed from " I/O first" to "computing first", and redesigned compact triple-level cache to reduce computing in list intersection and the snippet generation; (2) the idea of index structure changed from "memory space first" to " computing first", and redesign Bitfunnel + grammar index structure, using bit operation instead of list intersection, to reduce computing in list intersection. The motivation of this project comes from the enterprise practice, and the research results not only have academic value, but also can improve the level of Internet enterprises in China.

在互联网时代，搜索引擎是最重要的网络入口。在搜索引擎服务器部署3D XPoint SSD等新型存储设备之后，数据中心监控显示服务器的CPU负载显著升高，甚至可能影响服务器的安全运行。导致这一问题的原因是新型存储设备的高I/O性能使得搜索引擎服务器的系统瓶颈从存储I/O迁移到CPU计算。课题研究的目标是针对3D XPoint等新型存储设备的特点，重新设计搜索引擎缓存机制和索引结构，减少CPU计算任务。具体来说，课题研究内容包括：（1）将搜索引擎缓存设计思路从“计算换I/O”转变为“I/O换计算”，重新设计精简三级缓存结构，减少求交和摘要生成计算量；（2）将索引结构设计思路从“计算换存储空间”转变为“存储空间换计算”，重新设计Bitfunnel+文法的索引结构，用位与运算代替列表求交，减少求交计算量。课题研究动机来自企业实践，课题研究成果不但具有学术价值，也能够提升我国互联网企业的水平。

在以3D XPoint SSD为代表的NVM存储设备广泛应用与于数据中心后，NVM的高I/O性能使得搜索引擎服务器的系统瓶颈从存储I/O迁移到CPU计算。课题研究的目标是针对NVM的特点，重新设计搜索引擎缓存机制和索引结构，降低CPU负载。具体来说，课题研究内容包括：（1）设计实现基于NVM+DRAM的多级缓存搜索引擎架构，减少求交和摘要生成计算量；（2）设计Bitfunnel+文法的索引结构，用位与运算代替列表求交，减少求交计算量；设计基于有损压缩的位置索引构建，减少查询计算量。发表学术论文22篇，其中CCF推荐A类期刊、会议论文6篇，B类期刊、会议论文5篇。毕业博士生2人、硕士生5人。课题研究动机来自企业实践，课题研究成果不但具有学术价值，也能够提升我国互联网企业的水平。