术中立体内窥镜视频与超声图像增强现实系统关键技术研究

Endoscopic surgery is an attractive alternative to conventional open surgery and is known for improving outcomes, leaving fewer scars, and leading to significantly faster patient recovery. One challenging problem in current navigation systems is lacking the depth information of visualized structures due to the limitation of conventional flat displays. Both good depth perception and knowledge of internal structures are of critical importance for the safety and effectiveness of laparoscopic procedures and improved surgical outcomes. Using stereoscopic laparoscopes, surgeons can perceive true depth and get an improved understanding of the 3D spatial relationship among the visible anatomical structures, and using ultrasound, surgeons can visualize the internal anatomy that located beneath the visible organ surfaces. This project aims to develop augmented reality visualization for laparoscopic surgery and fuse intraoperative live endoscopic video with real-time ultrasonic images. The contents of the project include: designing stereoscopic augmented reality navigation surgery system for clinical use; establishing stereoscopic augmented-reality navigation mathematical model; proposing ultrasound image - stereoscopic video intraoperative registration algorithm; evaluating the accuracy and effectiveness of augmented reality system. The innovation of the project is developing a real-time stereoscopic augmented reality system for surgery by merging live ultrasound with stereoscopic video. Based on augmented reality system, surgeons can obtain good depth perception and knowledge of internal structures. Breaking the bottleneck of the existing image navigation systems, in which precision and speed of the preoperative/intraoperative multimodal registration technology is low, the proposed system can accomplish intraoperative real-time navigation surgery.

内窥镜手术具有出血少、恢复快等微创优势，已成为治疗多种疾病的标准术式。然而，传统内窥镜技术缺乏术中深度感知和内部结构信息，直接影响手术安全性和精准性。立体内窥镜为医生提供术中正确的深度感知信息，超声图像可提供深层组织的内部解剖结构，本项目拟利用新兴的立体内窥镜摄像技术，融合术中超声图像，通过术中超声图像与立体内窥镜视频的配准，将超声图像叠加显示到内窥镜视野中，使内窥镜产生实时“透视”的效果。本项目研究内容包括：手术场景中基于立体内窥镜手术导航系统设计；立体内窥镜手术导航系统统一数学模型研究；术中超声图像-立体内窥镜视频实时配准算法；术中增强现实算法的有效性、精确性评估。此项目的创新点：通过对术中内窥镜成像和超声成像两种不同模态的医学数据进行增强现实，使医生能够同时获得术中深度感知和内部解剖结构信息，突破现有内窥镜导航系统中术前/术中多模态配准技术中精度和速度的瓶颈，实现术中实时手术导航。

内窥镜手术具有出血少、恢复快等微创优势，已成为治疗多种疾病的标准术式。然而，传统内窥镜技术缺乏术中深度感知和内部结构信息，直接影响手术安全性和精准性。立体内窥镜为医生提供术中正确的深度感知信息，超声图像可提供深层组织的内部解剖结构，本项目拟利用新兴的立体内窥镜摄像技术，融合术中超声图像，通过术中超声图像与立体内窥镜视频的配准，将超声图像叠加显示到内窥镜视野中，使内窥镜产生实时“透视”的效果。本项目针对立体内窥镜手术，开展基于超声图像实时配准的手术导航系统理论和方法研究。通过建立基于术中超声图像实时配准的立体内窥镜导航新方法，为医生提供动态实时的三维解剖结构信息和空间定位功能。具体研究内容如下： A．设计适用于实际手术场景的立体内窥镜手术导航系统。能将术中病人的超声图像与立体内窥镜视频融合显示，从而提供病人组织结构之间正确的空间深度信息和内部的解剖结构信息。B．通过对术中超声成像系统、立体内窥镜摄像系统、图像配准、术中定位和增强现实导航等各子系统参数进行统一数学描述，建立基于超声图像实时配准的立体内窥镜手术导航系统统一模型。C．在上述超声图像、立体内窥镜视频标定的基础上，基于同一时刻下超声图像与立体内窥镜视频中左右视图的空间位置关系，建立两种模态图像的空间仿射变换关系，实现超声图像-立体内窥镜视频的实时配准。D．通过体模实验、动物实验，完成客观指标和主观评测的评估，从而为立体内窥镜手术提供新的解决方案。本项目从立体内窥镜手术的临床需求出发，在总结现有研究中普遍存在的问题的基础上，开展基于术中超声图像的增强现实导航研究，为立体内窥镜手术提供更为精确安全的新解决方案。本项目在立体内窥镜手术导航的理论和实践层面，都具有重要意义。