面向短文本理解的带约束语义文法自动学习方法研究

Short texts understanding is crucial to many applications, such as speech understanding, semantic search etc. Short texts have some unique characteristics such as not always observing the syntax of written texts, less context and more ambiguous, which make them difficult to handle. Upon an existing natural language understanding system based on domain ontology and constrained semantic grammar, this research is to further study and realize the method of automatic learning of constrained semantic grammar. Firstly, a semantic grammar rule extending algorithm will be studied to solve domain coverage problem, which first partially parses the (ungrammatical) sentences based on the core semantic grammar, and attempts to build a complete parse tree, including predicting the top-level node of the partial parsing tree, generating and abstracting to learn new grammar rules. Secondly, constraints learning is regarded as an inductive logic programming problem, and we will focus on optimizing search strategy, search stop criterion and learning stop criterion. Lastly, the evaluation of grammar learning includes evaluation of grammar rules and constraints, we will study on redundant detection and ambiguity detection based on rule expansion set and evaluating constraints based on constraint classification and complexity. This research is an important foundation for further study of domain-independent system.

短文本理解是口语理解、语义搜索等应用的核心技术之一。短文本通常具有不符合书面语法、语境少歧义多等特点，本课题拟在已构建的一个基于领域本体和带约束语义文法的自然语言理解系统基础上，进一步研究和实现语义文法自动学习方法，包括：拟研究一种文法规则自动扩展学习方法，首先利用核心文法对解析失败句子进行部分解析，基于部分解析树预测顶层节点及子节点，基于相似性度量等进行规则扩展并构建完整解析树，经过垂直概化和平行概化处理后得到新文法规则；针对文法歧义问题，拟研究一种文法约束自动学习方法，将其看作一个ILP学习问题，通过改进搜索策略、约束搜索停止准则、约束学习停止准则等以保证高效地学习到可以覆盖尽量多正例且覆盖尽量少反例的约束；为了保证文法学习质量，拟研究基于文法规则扩展集的冗余检测和歧义检测方法以及基于约束划分效果和约束复杂度对文法约束进行评价。

针对短文本通常具有不符合书面语法、语境少歧义多等特点，本课题在已构建的一个基于领域本体和带约束语义文法的自然语言理解系统基础上，研究和实现了一种基于错误驱动的语义文法自动扩展学习方法，首先利用核心文法对解析失败句子进行部分解析，基于部分解析树预测顶层节点及子节点，基于相似性度量等进行规则扩展并构建完整解析树，经过垂直概化和平行概化处理后得到新文法规则；针对文法歧义问题，研究和实现了一种基于ILP的文法约束自动学习方法，通过改进搜索策略、约束搜索停止准则、约束学习停止准则等以保证高效地学习到可以覆盖尽量多正例且覆盖尽量少反例的约束；为了保证文法学习质量，研究和实现了基于文法规则扩展集的冗余检测和歧义检测方法以及基于约束划分效果和约束复杂度对文法约束进行评价。测试对比了提出的算法在不同规模领域的学习效率，以及更新后的文法和核心文法在两个领域数据集上的准确率和识别率等性能指标，实验结果表明，本项目所提出的文法扩展学习和约束学习方法是有效的。研究成果在诸多应用领域，如智能客服、智能信息检索、智能推荐系统等方面，均有着潜在的应用价值。