版式文档中复杂异构对象的识别技术研究

With the development of digital publishing in the era of mobile internet, to solve the difficulty on recognition of tables, formulas, graphs and other complex and heterogeneous page objects is becoming an urgent demand. On the other hand, the traditional layout analysis techniques mostly focus on image documents, thus have not utilized the rich description information of low-level page elements that existing in the fixed-layout documents such as PDF documents. Therefore, this project focuses on the research of location and structure analysis of complex and heterogeneous objects in the fixed-layout documents, and will conquer the diversity of document formats and the differences of underlying information quality between different documents. Utilizing the accurate character and style information, vector graphics features, this project will propose the method of page object location based on multi-source information fusion(visual information, characters, graphics, context, feedback, etc) and deep learning, and the method of object (formula, table, graph) recognition based on statistical machine learning and grammar analysis. In addition, this project will establish a global page analysis of various kinds of page objects and a feedback mechanism of page object location and structure analysis, to optimize the final result of page object recognition. Based on the accumulation of fixe-layout document processing technologies and the novel methods proposed in the project, we will establish a new and efficient platform of fixed-layout document analysis and recognition to support digital publishing, online education, mobile reading and other industries.

随着移动互联网时代数字出版的变革，亟需解决文档版面中表格、公式和图形等复杂异构对象的识别难题。传统的版面分析技术主要对文档图像进行分析，缺乏对PDF等主流版式文档丰富的底层图元描述信息的利用。本课题针对版式文档，研究其复杂异构对象的自动定位与结构分析，克服版式文档格式多样化、底层信息质量差异化等预处理难题，利用版式文档中字符和样式信息准确丰富、矢量化描述图形图像等新特性，提出基于多源信息融合（视觉、文字、图形、上下文、反馈等）和深度学习的对象定位方法、结合统计学习与语法分析的对象识别（含公式识别、表格识别、图形识别）方法，同时建立多种复杂异构对象的全局分析和识别过程反馈机制，优化版面对象的识别效果。基于上述创新方法和版式文档技术的历史积累，构建高效的版式文档分析与识别技术平台，服务于数字出版、教育与移动阅读等行业。

以PDF为代表的版式文档已取代图像文档成为数字出版物的主要来源，版面中表格、公式和图形等复杂异构对象的识别，是文档识别领域的研究难点。为此，本项目针对版式文档，研究了公式识别、表格识别、复杂版面分析等关键科学问题，提出了基于多源信息融合（视觉信息、文字流、图形流、上下文信息、结构分析结果等）和深度学习的复杂版面对象定位方法，结合统计学习与语法分析的复杂版面对象结构分析，基于全局分析与反馈优化的版面对象识别方法，克服了传统的文档图像识别受限于不可靠输入和信息匮乏所存在的性能瓶颈。本研究团队也成长为国际公认的版式文档识别主要研究小组之一；以此研究成果为基础，在模式识别领域的重要会议ICDAR2017上举办了版面对象识别国际竞赛，在ICDAR2019上主办了表格识别竞赛；发表论文11篇，申请发明专利3项，获软件著作权1项，已经实际应用于2个电子书和题库系统之中，在题库构建、在线教育学习、知识服务等方面，表现出很好的应用前景。