NAVMR - Needle insertion NAVigation with Mixed Reality

This repository contains the source code for the applications developed for the paper: A Comparative Study of MR-Guided Needle Insertion for Surgical Procedures: Insights from HoloLens 2, Magic Leap 2, and Apple Vision Pro.

Overview

This project includes two separate applications that provide mixed-reality (MR) guidance for a Sacral Neurostimulation (SNS) needle insertion procedure.

Application functionality

The goal of the applications is to guide a clinician in accurately inserting a needle through the S4 right foramen of the sacrum.

Both apps function by detecting a single physical registration marker (e.g., a Vuforia Image Target) placed on a surgical phantom. Once the marker is detected, the following virtual content is registered and anchored to it:

Anatomical Models:

• The skin of the physical phantom.
• The sacrum bone structure.

Guidance Cues:

• The needle trajectory line.
• A helper cone around the trajectory to assist in aligning the needle's angle.
• A depth cylinder that indicates the correct final insertion depth for the needle.

Utility & Assessment:

• A virtual model of the registration marker (used to visually assess registration accuracy).
• A floating menu that allows the user to toggle the visibility of each virtual model.

Technical Implementations

This repository contains two parallel implementations of this functionality for different ecosystems:

1. A cross-platform application for Magic Leap 2 and HoloLens 2, developed in Unity (v2022.3.30f1) with the Vuforia Engine (v11.0.4).

2. A native application for Apple Vision Pro, developed in SwiftUI and native Apple frameworks (ARKit, RealityKit).

Getting Started

To build and run one of the applications, please navigate to the corresponding folder and follow the instructions in its local README.md file:

• For Magic Leap 2 & HoloLens 2: Go to /NAVMR-Unity/README.md
• For Apple Vision Pro: Go to /NAVMR-SwiftUI/README.md

Cite this work

Amaia Iribar-Zabala, Joseba Ruiz-Olalla-Del-Fresno, Inés Rubio-Pérez, et al. A Comparative Study of MR-Guided Needle Insertion for Surgical Procedures: Insights from HoloLens 2, Magic Leap 2, and Apple Vision Pro. Authorea.