基于协同注意力机制与病理属性挖掘的可解释性肿瘤图像诊断模型研究

Tumor is one of the most serious diseases which could threaten people's health in the world. Recently, with the rapid development of many modern computer technologies such as artificial intelligence (AI), computer vision (CV) and computer hardware, the cell pathological diagnosis based on lesions tissue sections plays a more important role in tumor image diagnosis than before. However, several key problems still remain in the traditional diagnosis models. First, the discriminant abilities of the image features extracted by the traditional diagnosis model are weak, which could affect final recognition performance and heavily decrease the practical value of the diagnosis model. Second, the interpretable abilities of the diagnosis results generated by the traditional diagnosis model are low, which can’t meet the higher demanding of those professional pathological doctors. In actual diagnosis works, the pathological doctors need more useful or valuable diagnosis results that contain lots of pathological semantic information rather than those discrete recognition results such as “0” (normal) or “1” (cancer). To overcome these problems in some extent, we obey the knowledge of pathology and firstly design a novel feature learning model which combines the traditional image features, the state-of-art deep learning features, and the late-fusion algorithms together. By using the feature learning model, a group of new image features can be acquired more easily. They focus on describing the key visual content of the tumor images more accurately than before. Based on the extracted new image features, we create a novel pathological attributes mining (PAM) model to obtain those valuable pathological attributes in the tumor images and use them to better analyze the deep-level pathological semantic information in the tumor images. The pathological attributes mined by the PAM model will contribute to enrich the semantic content of the final diagnosis reports. Based on the extracted new image features and the mined pathological attributes, the state-of-art co-attention mechanism is in turn introduced to analyze the multi-modal distributional semantic between the tumor images and the diagnosis reports. The novel strategy will help to give more attention on those key lesion areas in the tumor images and generate more useful diagnosis results for the professional pathological doctors. Finally, based on the above works, a new natural language generation (NLG) model which absorbs the multi-modal distributional semantic into it is built. And the coherent diagnosis reports of a lot of valuable pathological semantic information are generated by the novel NLG model. We hope our research works can really improve the interpretable ability of the tumor image diagnosis model and it can also detect the illness more early and more accurately than before. Moreover, we also hope our works will better assist the clinical pathologic works of the professional pathological doctors.

肿瘤是威胁人类健康最严重的疾病之一。伴随人工智能、机器视觉、计算机硬件等技术的发展，基于病变组织切片的细胞病理学分析在肿瘤图像诊断中发挥了重要作用。现有研究存在“特征判别性较弱”、“诊断结果的可解释性不高”等问题，无法满足病理医生对更丰富诊断结果的需求。针对这些问题，拟紧扣病理学知识，基于传统图像特征、深度学习特征及后融合算法设计新的特征学习模型，以抽取一组能准确刻画肿瘤图像的新特征；构造病理属性挖掘模型，深入分析肿瘤图像中的病理语义信息，以丰富诊断报告内容；基于协同注意力机制分析肿瘤图像、诊断报告之间的多模态分布式语义，以聚焦图像中关键病灶区域；最后，设计融入多模态分布式语义的自然语言生成模型，输出连贯、流畅且包含丰富病理语义信息的诊断报告。期望本研究能提升肿瘤图像诊断模型的可解释性，从而有助于及早地发现病症，更准确、高效地辅助病理医生的临床诊断工作。

肿瘤是威胁人类健康最严重的疾病之一。传统人工筛查多依赖临床诊断经验，其效率低下且假阳率偏高，模型的实用性不高。基于模式识别、医学影像处理、机器视觉、深度学习等主流人工智能（Artificial Intelligence，简称AI）技术的计算机辅助诊断（Computer-Aided Diagnosis，简称CAD）系统是破解这一困局的有效手段。然而，现有肿瘤图像诊断研究存在“高质量标注样本稀缺”、“深层病理信息未得到有效挖掘”、“异构网络间相关性未有效利用”、“诊断模型的可解释性不高”等关键问题，严重制约CAD系统的实用价值。针对这些问题，本项目在充分了解领域内热点问题、先进技术、流行方法的基础上，从如下七个方面开展深入的研究工作：1）基于在线互知识传递模型融合的乳腺癌病理图像分类模型MF-OMKT。通过双向知识传递提升模型性能。2）基于深度对比互学习的新冠肺炎图像识别模型DCML。DCML将对比学习思想融入互学习之中，学习更好的图像特征。3）基于嵌入式融合互学习的乳腺癌病理图像分类模型EFML。EFML模型联合特征蒸馏和逻辑层蒸馏改善诊断精度。4）融入深层病理信息挖掘的乳腺肿块识别模型。设计多层特征优选方法，优选判别性强的图像特征，提高诊断精度。5）面向可解释性工作的多视角注意力引导的多示例检测网络MA-MIDN。通过多种注意力引导模型同时完成病理诊断和病灶区域定位。6）基于多尺度空洞卷积网络（MMDC-Net）的眼底图像分割模型。联合多尺度的空洞卷积扩大分割模型的感受野。7）基于TiM-Net的眼底图像分割模型，将Transformer融入跳跃连接，以捕获特征中的远程依赖。基于上述研究工作，课题组发表高水平论文28篇，其中，完成SCI检索论文18篇，国内权威EI检索4篇；撰写综合研究报告1份；授权软件著作权5项；授权发明专利6项；培养10名硕士研究生取得学位，其中1人获省优秀硕士学位论文称号，5人获校级优秀硕士学位论文称号，指导在读研究生16名。综上，项目组已高质量完成合同书中各项经济、技术指标。本项研究成果可准确、高效地辅助医生的临床诊断活动，有力地推动了医工深度融合。