基于强化学习的信息检索排序模型研究

With the rapid growth of Internet, represented by the mobile Internet and information feeds, the gap between the users' dynamic, precise, and diverse information needs and the massive web data is becoming more and more serious. To address the issues and to improve the search experience, one key problem is to track the users’ latent information need so that the web search engines can capture the dynamic information needs during the interactions of the users and search engines, and thus can provide users more accurate and diverse web search results. In recent years, the breakthrough of the deep reinforcement learning models and applications provided an opportunity for addressing the above issues. However, the user activities are complex and the user’s information needs always changing during the search process, which makes it difficult to directly apply the deep reinforcement learning algorithms to web search. In this project, we aim to solving the key scientific challenges of tracking and modeling of the dynamic user information needs, avoiding of the suboptimal webpage rankings in search, and the cooperation of the multiple agents in multi-agent learning to rank model. The research focuses on reinforcement learning to rank for interactive search modeling, reinforcement learning to rank for search result diversification, multi-agent learning to rank, and distributed computing platform for reinforcement learning to rank. The goal is to develop novel refinement learning to rank models for web search, to provide a systematic theoretic foundation and to provide key technologies for IR, to push the applications of reinforcement learning to web search and related area, and finally to advance the research of web search and information retrieval in China.

随着以移动互联网、信息流为代表的互联网的高速发展，用户精准而多样化的信息需求与庞大的互联网数据之间存在的信息鸿沟愈发明显，如何在用户与搜索引擎交互过程中跟踪用户动态的信息需求、提供精准、多样化的综合搜索结果以降低用户获取信息的代价，是弥合信息鸿沟、提升用户体验的关键难题。近年来人工智能中深度强化学习技术的突破为解决该问题提供了契机。然而搜索过程中用户搜索行为复杂、需求多变的特性，都给强化学习技术在互联网搜索中的应用提出了全新的挑战。针对上述挑战，本课题围绕用户动态信息需求跟踪与建模、搜索中局部最优排序的避免、多排序智能体的协同三个科学问题，从交互式搜索模式、多样化排序学习模型、多智能体协同排序和大规模强化排序学习计算平台四个层面展开研究，旨在建立面向网络搜索的强化排序学习模型，建设成体系的基础理论和关键技术，推动强化学习技术在互联网搜索行业的应用，促进我国网络搜索技术的研究发展。

深入分析用户的检索行为、识别用户的搜索意图并进一步提供更好的搜索结果来满足用户的信息需求，是工业界和学术界广泛关注的研究问题。本课题针对信息检索过程中用户搜索行为复杂、需求多变的现象给强化学习技术在互联网搜索中的应用中所提出的挑战，围绕用户动态信息需求跟踪与建模、搜索中局部最优排序的避免、多排序智能体的协同三个科学问题，在多样化排序学习模型、交互式搜索算法、大规模强化排序学习模型与系统、查询-文档语义匹配四个方面形成了突破，形成了包括低秩排序支持向量机、成对策略梯度法、深度语义匹配模型、基于强化学习的梯度量化方法等一系列原创性的理论方法与模型，在国内外重要会议和期刊发表(录用)学术论文57篇、撰写英文学术专著1本、授权国内发明专利1项、申请国内发明专利8项、申请国际发明专利1项，获得省部级奖励1次（二等奖、排名第2）、国际学术奖励1次、国内学术奖励1次、人民大学优秀科研成果奖1次，指导学生获奖3次。通过本课题的研究，能有效解决利用大规模用户行为数据训练搜索模型所面临的核心问题，进一步提升信息检索技术的性能，推动信息检索系统在互联网中更为广泛有效的使用。