AI-Driven Qualitative Skill Assessment in Laparoscopic Training: A Prospective Observational Study

Background: Qualitative assessment of surgical skills is essential for proficiency-based training but remains resource-intensive and difficult to scale due to reliance on expert observation. Automated simulator metrics, while objective, are largely limited to quantitative parameters and fail to capture qualitative aspects of surgical performance. Artificial intelligence–based video analysis may address this gap by enabling standardized and reproducible qualitative assessment. Methods: In this prospective observational study conducted at the 41st Annual Davos Surgical Course (2024), 50 first- and second-year surgical residents performed laparoscopic cholecystectomy on porcine simulation models. Procedures were video recorded, anonymized, segmented, and independently assessed by expert raters and an AI-based model using the Global Operative Assessment of Laparoscopic Skills (GOALS). AI test–retest reliability and agreement with expert ratings were evaluated. Results: AI demonstrated excellent test–retest reliability for the total GOALS score (ICC 0.91) and good reliability across most domains. Agreement between AI and expert raters was excellent for the total score (ICC 0.92) and good to excellent for individual domains, with minimal bias on Bland–Altman analysis. Conclusion: AI-based video analysis enables reliable, reproducible qualitative assessment of laparoscopic surgical skills in a simulation setting and may support scalable integration of qualitative evaluation into surgical training programs and other video-based clinical skill assessment domains.