Advancing Dense Endoscopic Reconstruction with Gaussian Splatting-driven Surface Normal-aware Tracking and Mapping
Simultaneous Localization and Mapping (SLAM) is essential for precise surgical interventions and robotic tasks in minimally invasive procedures. While recent advancements in 3D Gaussian Splatting (3DGS) have improved SLAM with high-quality novel view synthesis and fast rendering, these systems struggle with accurate depth and surface reconstruction due to multi-view inconsistencies. Simply incorporating SLAM and 3DGS leads to mismatches between the reconstructed frames. In this work, we present Endo-2DTAM, a real-time endoscopic SLAM system with 2D Gaussian Splatting (2DGS) to address these challenges. Endo-2DTAM incorporates a surface normal-aware pipeline, which consists of tracking, mapping, and bundle adjustment modules for geometrically accurate reconstruction. Our robust tracking module combines point-to-point and point-to-plane distance metrics, while the mapping module utilizes normal consistency and depth distortion to enhance surface reconstruction quality. We also introduce a pose-consistent strategy for efficient and geometrically coherent keyframe sampling. Extensive experiments on public endoscopic datasets demonstrate that Endo-2DTAM achieves an RMSE of 1.87±0.63 mm for depth reconstruction of surgical scenes while maintaining computationally efficient tracking, high-quality visual appearance, and real-time rendering.
同时定位与建图（SLAM）对于精确的外科手术干预和微创手术中的机器人任务至关重要。尽管 3D 高斯 Splatting（3DGS）最近的进展已通过高质量的新视角合成和快速渲染提升了 SLAM 的性能，但这些系统在深度和表面重建上仍面临挑战，主要由于多视角之间的不一致性。简单地将 SLAM 和 3DGS 结合会导致重建帧之间的不匹配。为了解决这些问题，我们提出了 Endo-2DTAM，一种基于 2D 高斯 Splatting（2DGS）的实时内窥镜 SLAM 系统。Endo-2DTAM 引入了一个表面法线感知的流程，包括跟踪、建图和束调整模块，以实现几何精确的重建。我们的稳健跟踪模块结合了点对点和点对平面距离度量，而建图模块则利用法线一致性和深度失真来提高表面重建质量。我们还提出了一种姿态一致性策略，用于高效且几何一致的关键帧采样。基于公共内窥镜数据集的大量实验表明，Endo-2DTAM 在外科场景的深度重建中实现了 1.87±0.63 毫米的 RMSE，同时保持了计算高效的跟踪、高质量的视觉效果和实时渲染能力。