A Local Outpatient Practice-Level Prediction Model for Short-Term Psychiatric Emergency Presentation
Discuss this preprint
Start a discussion What are Sciety discussions?Listed in
This article is not in any list yet, why not save it to one of your lists.Abstract
Importance
Psychiatric emergency department (ED) presentations are difficult to predict using general medical risk stratification tools. Health information exchange (HIE) data may improve prediction by capturing fragmented care across settings.
Objective
To develop and temporally validate a machine learning model using HIE and geospatial data to predict 30-day psychiatric ED presentation among outpatients receiving psychiatric care and to compare its performance with standard clinical risk scores.
Design, Setting, and Participants
This retrospective cohort study included patients seen at Frontier Psychiatry with records in the Big Sky Care Connect statewide HIE. Structured clinical data were linked to zip code–level sociodemographic measures. The analytic unit was the patient snapshot, defined as all structured data available up to a given point. Models were evaluated in temporally separated train and test sets.
Exposures
Predictors derived from HIE structured data, including prior utilization, diagnoses, medications, laboratory data, and zip code–linked geospatial deprivation and vulnerability measures.
Main Outcomes and Measures
The primary outcome was psychiatric ED presentation within 30 days, identified from structured encounter-type fields and primary diagnosis codes for psychiatric or substance use disorders. Model discrimination was compared with a parsimonious clinical baseline model and LACE and Elixhauser scores.
Results
In the test set, 343 of 16,469 snapshots (2.1%) were followed by a qualifying psychiatric ED presentation within 30 days, corresponding to 102 ED visits among 68 patients. The machine learning model showed discrimination in temporally held-out testing and outperformed the clinical baseline model as well as LACE and Elixhauser scores. At a prespecified decision threshold, the model reduced the number needed to evaluate from more than 40 with universal screening to 3.4 to identify 1 true-positive case, while identifying over two fifths of 30-day psychiatric ED presentations.
Conclusions and Relevance
In this retrospective cohort study, a locally developed machine learning model using statewide HIE data showed improved prediction of 30-day psychiatric ED presentation compared with selected general-purpose risk scores. The results support the feasibility of HIE-enabled local psychiatric risk modeling and suggest other practices could develop similarly tailored models. Prospective studies are needed to assess clinical utility and effects on outcomes.
