Computer Vision for Real-Time Pixel-Level Anatomical Segmentation in Neurosurgery: First-in-Human Clinical Evaluation and Iterative Development (IDEAL Stage 1)
Discuss this preprint
Start a discussion What are Sciety discussions?Listed in
This article is not in any list yet, why not save it to one of your lists.Abstract
Introduction
Precise anatomical navigation is fundamental to safe endoscopic pituitary surgery, a high-stakes procedure characterised by a challenging learning curve. While traditional navigation systems often rely on workflow-disrupting probes or static preoperative imaging, advancements in computer vision AI (CVAI) now enable dynamic, real-time pixel-level anatomical segmentation directly from live surgical video. Our group has previously conducted a series of preclinical human-computer interaction studies to refine the system’s design, alongside digital and high-fidelity physical simulations demonstrating the benefit of AI assistance in improving overall performance, training, and safety 4–8 . Building on this foundation, the current study represents a first-in-human application of real-time pixel-level CVAI anatomical segmentation in the neurosurgical operating room, serving to assess feasibility and to iteratively improve the system.
Method
Guided by DECIDE-AI and IDEAL frameworks, this single-centre evaluation comprises an initial proof-of-concept phase (n=6) for endoscopic transsphenoidal pituitary surgeries. The AI model utilised a DINOv3-derived vision transformer architecture, deployed via a high-performance edge computing unit to achieve low-latency, real-time inference without reliance on cloud infrastructure. Feasibility and functionality were assessed via structured questionnaire, prospective observation, and blinded retrospective review of the recordings of the endoscopic surgical video feed and wider operating room environment. Continuous multi-stakeholder feedback through validated human factors surveys drove iterative technical refinements between cases.
Results
Eight patients with pituitary adenomas were enrolled. The CVAI system was successfully deployed in six cases, demonstrating acceptable real-time pixel-level sella segmentation accuracy. Deployment failed pre-operatively in two cases owing to a single recurring system reboot bug. Iterative refinement between cases were driven by our experience and surgical team feedback. This resulted in the integration of additional anatomical structure segmentations (e.g., carotid arteries), enhanced model accuracy via training dataset expansion, and hardware firmware upgrades. Multi-stakeholder surveys demonstrated satisfactory system feasibility, usability, and acceptability among the surgical team. Both prospective observation and retrospective video review confirmed the absence of adverse events, including no significant distraction to the primary surgeon, and there were no AI-related clinical complications.
Conclusion
This first-in-human early clinical evaluation demonstrates the feasibility and iterative development of real-time pixel-level CVAI-based anatomical navigation during high-stakes neurosurgery. Future work will include a larger single-centre case series (IDEAL Stage 2a) with more surgical teams to further iterate the system and explore its impact on safety, training and workflow. As the underpinning AI models improve and integrate with other intra-operative navigational technologies, such tools will likely be the cornerstone of intra-operative surgical decision support systems.
