Towards Mixed Reality-Navigated Surgery: Point Cloud Surface Registration for Automated 3D Image Overlay

Background Mixed Reality (MR) enables intuitive and tangible visualization of patient data in interventional environments without further exposure to radiation, while most state-of-the-art navigation systems lack proper immersive visualization and rely on the use of markers as well as additional intrasurgical imaging for digital-physical registration. Methods We develop and validate a modular and immersive system combining state of the art mixed reality hardware, high-fidelity tracking/scanning technology, and automated surface registration algorithms to generate an accurate MR holographic projection of surgical tools and patient anatomy. Results The system exhibits reliable real-time performance with latencies below 50 ms. The accuracy of the holographic projection was measured for registration and MR precision, revealing a combined average deviation of 1.73 mm on a patient model. Conclusion Overall system accuracy provides a strong foundation for immersive surgical or interventional guidance. Ongoing studies aim to investigate clinical value and usability in real-world scenarios.