Human Factors Validation Study of an Artificial Neural Network‑based Preoperative Decision‑support Tool for Noninvasive Lymph Node Staging (NILS) in Women with Primary Breast Cancer (ISRCTN99301435)

Importance The integration of clinical decision support tools in medical practice is challenging and must be carefully undertaken, especially in cancer management. Noninvasive Lymph Node Status (NILS) is a web-based tool designed to estimate the probability of healthy axillary lymph nodes in female patients with breast cancer scheduled for primary surgery. Objective To identify barriers to NILS adoption in a clinical setting, and assess whether intended users can operate the tool without significant errors or difficulties. Design This mixed-methods qualitative study used simulated clinical cases, the System Usability Scale (SUS), and the After-Scenario Questionnaire (ASQ) to evaluate usability and satisfaction. An oral interview was conducted after the evaluations. The study followed a structured protocol, with distinct roles assigned to the test participants, leader, and observer. Setting A multicenter on-site usability study was conducted in a simulated clinical environment, replicating both real-world physical and digital conditions. Participants Based on the identified target user population for NILS, twenty physicians comprised the cohort. Exposure Web interface interaction using simulated clinical cases. Main Outcomes and Measures Physicians' perceptions of the NILS as a decision-support tool for determining whether to perform or abstain from sentinel lymph node biopsy in a simulated clinical setting. Results Twenty physicians (15 [65%] female; 15 [75%] surgeons; 5 [25%] oncologists; median 9.5 years in specialist-practice) working in four different hospitals participated. Usability scores were high, with a mean SUS score of 89.5 (“excellent”) and a mean ASQ score of 6.3 (Likert scale 1-7). Several interface challenges were identified, including difficulty in locating the reset and information buttons, the ability to enter values outside valid ranges (age and tumor size), and the risk of unintentionally modifying the entered values while scrolling. Conclusions and Relevance This study identified the key usability factors, barriers, and facilitators affecting NILS implementation. Physicians found the NILS interface easy to use and valued the presented risk estimates of healthy axillary lymph node status in their decisions to perform or abstain from sentinel lymph node biopsy. Redesign initiatives are ongoing. An iterative process of addressing usability in a clinical setting is crucial for successful implementation. Trial Registration ISRCTN99301435