An Experimental Investigation of the Relationship between AI-Human Workflow Design and Legal Liability for Radiologists: The Erroneous-Change Penalty and Omission Bias
Discuss this preprint
Start a discussion What are Sciety discussions?Listed in
This article is not in any list yet, why not save it to one of your lists.Abstract
Background
With growing impetus to integrate artificial intelligence (AI) tools into radiology, clinical practices must navigate workflow redesign. This carries implications for medical malpractice liability.
Methods
We conducted an online vignette experiment with United States adults who acted as hypothetical jurors in a malpractice case involving a missed intracranial hemorrhage. Participants (n=2,347) were randomized to one of 22 conditions: a no-AI control and 21 conditions involving a hypothetical AI system. These twenty-one conditions varied by whether (1) a single-read or double-read workflow was used, (2) the radiologist’s initial interpretation was documented, (3) the radiologist changed their interpretation after viewing AI output, (4) the AI detected the abnormality, and (5) the AI error rate—False Discovery Rate (FDR) or False Omission Rate (FOR)—was provided to participants only, both participants and radiologist, or neither. The primary outcome was perceived liability, assessed by whether the radiologist met their duty of care.
Findings
Perceived liability differed across conditions (p<0.0001). Double-read workflows (p<0.0001), documenting initial interpretations (p=0.0125), and providing participants with AI error rates, including the FDR (p=0.0038) or FOR (p=0.0035), reduced perceived liability. Liability was also lower when AI was incorrect (p<0.0001). Radiologists’ awareness of AI error rates did not significantly impact liability. Notably, we observed an “erroneous change penalty”: the greatest liability occurred when radiologists initially identified an abnormality but later changed their interpretation to normal after seeing that AI identified the case as normal; conversely, perceived liability was lowest with documented, double-read workflows.
Interpretation
Double-read workflows with documented initial interpretations and disclosure of AI error rates reduce perceived liability, though changing a correct initial interpretation increases it. Strategic workflow design is critical for successful AI implementation that can mitigate malpractice risk.
RESEARCH IN CONTEXT
Evidence before this study
Emerging research has identified several factors that shape how artificial intelligence (AI) systems are integrated into clinical workflows. Beyond technical performance, factors such as disease prevalence, documentation practices, and workflow design have all been shown to play a role in the implementation of AI tools and how they will inevitably affect physician liability. Vignette experiments have separately identified cognitive biases and mitigating measures that can be integrated into workflow design, such as disclosing AI error rates and documenting independent interpretations before reviewing AI output.
Added value of this study
This study builds on prior work by examining how multiple aspects of AI implementation influence perceptions of legal liability when hypothetical jurors are asked to adjudicate a medical malpractice lawsuit arising from a false negative interpretation. In particular, we show how combining double-read workflows, documentation practices, and the inclusion of AI error rates can reduce perceived liability. We also show that, when a double-read workflow is used, changing an initially correct interpretation to an incorrect one incurs greater liability for the radiologist than being incorrect in both the initial and final interpretations. These findings underscore the need to address cognitive biases that will undoubtedly arise at the human-AI interface.
Implications of all the available evidence
Optimizing the integration of AI tools into radiology requires strategic attention to workflow redesign, as combinations of features can collectively affect perceived liability, likely through well-known cognitive biases. Based on the results of our study, we propose one workflow that can mitigate a radiologist’s risk of legal liability. Moving forward, clinical practices and stakeholders should remain cognizant of these factors as they work toward building sustainable AI-physician systems.
