Assessing the Proficiency of ChatGPT in Image-Based Robot-Assisted Radical Prostatectomy Scenario: Comparative Study

Background Many surgical teaching and assistance applications based on artificial intelligence have emerged. However, the application of the newly released multimodal ChatGPT 4o in surgical contexts remains unexplored. This study aims to investigate the proficiency of ChatGPT 4o in a simulated surgical context and compare the ability of ChatGPT 4o with urologists and medical students to verify its educational significance and application potential in surgical field. Methods Utilizing screenshots from robot-assisted radical prostatectomy videos sourced from the internet, we developed a test called “RARP Simulation Test” and selected 20 questions from them as a quiz called “RARP Knowledge Quiz”, both including sections on anatomical structures, surgical procedures, operational precautions and case reports. We first test ChatGPT 4o with RARP Simulation Test. Then, the RARP Knowledge Quiz was given to 35 urologists, 35 medical students and ChatGPT 4o for comparison, respectively. Results In the RARP Simulation Test, ChatGPT 4o achieved an average accuracy rate of 81.3%, reaching the excellent threshold, and maintained consistent performance across different categorizations. In the RARP Knowledge Quiz, ChatGPT 4o achieved a performance comparable to that of urologists and far superior to that of medical students. Conclusions ChatGPT 4o demonstrated comprehensive and highly stable capabilities in RARP surgery, encompassing preoperative diagnosis, perioperative procedures, and postoperative considerations, reaching the proficiency level of urologists. ChatGPT exhibits significant potential in operative assistance and surgical education field.