Considerations for implementing a machine learning model for suicide risk in the health systems that serve American Indian communities
Listed in
This article is not in any list yet, why not save it to one of your lists.Abstract
Objective
We explored the implementation considerations of machine learning (ML) suicide risk identification in a health system that serves primarily American Indian patients in the Southwestern United States. Previous work developed and reported the performance of suicide risk models, but traditional performance metrics do not directly reflect clinical workflows.
Materials and Methods
Using EHR data collected for Emergency Department (ED) visits between 10/01/2019 and 10/02/2021, we conducted Decision Curve Analysis (DCA) to identify the standardized net benefit (SNB) of all models previously evaluated and plotted the potential workload implications of the model with the greatest SNB. Finally, we evaluated suicide screening practices during the study period to inform implementation barriers.
Results
Among 9,244 patients, 139 (1.5%) had a suicide attempt or death within 90 days of an included visit. The population-specific logistic regression model had the greatest SNB compared to other approaches. When operationalized, the 95 th percentile cutoff did not substantially increase clinical workloads, while the 75 th percentile cutoff substantially impacted workloads. The secondary analysis on suicide screening depicted variations in screening administration over time.
Discussion
While DCA identified advantages of ML suicide risk identification over current standards, the complexities of suicide-related care limited its clinical interpretability. A stepped-care framework may mitigate potential workload burdens. Training initiatives and dynamic implementation strategies will support high-quality implementation of ML suicide risk identification.
Conclusion
Our analyses support the feasibility and utility of augmenting existing suicide screening practices with ML suicide risk identification, and demonstrate the importance of nuanced analyses to inform implementation considerations.
