Development and External Validation of a High-Precision Model for Predicting ICU Admission from Emergency Department Triage
Listed in
This article is not in any list yet, why not save it to one of your lists.Abstract
Objective
To develop, internally evaluate, and externally validate a machine-learning (ML) model predicting intensive care unit (ICU) admission or death using information available solely at emergency department (ED) triage. Performance was primarily assessed by area under the precision-recall curve (AUPRC) to address severe class imbalance.
Methods
We trained an XGBoost classifier on the Medical Information Mart for Intensive Care IV (MIMIC-IV) dataset. Positive outcomes were ICU admission or death within 6 hours of arrival. Features included vital signs, engineered physiological measures, clinician-assigned acuity, demographics, chief complaint, and home medications. Model performance was internally evaluated through group-stratified five-fold cross-validation and externally validated on the Multimodal Clinical Monitoring in the Emergency Department (MC-MED) dataset.
Results
In the internal validation (MIMIC-IV, 350,241 visits; 11,745 ICU/death), the model achieved an AUPRC of 0.736 (95% CI: 0.728–0.743), AUROC of 0.966 (95% CI: 0.965–0.968), and accuracy of 0.936 (95% CI: 0.936–0.938). On external validation (MC-MED, 42,624 visits; 1,503 ICU/death), the model retained robust performance with an AUPRC of 0.602 (95% CI: 0.578–0.624, 0.134 decrease), AUROC of 0.949 (95% CI: 0.944–0.955, 0.017 decrease), and accuracy of 0.928 (95% CI: 0.927-0.932, 0.007 decrease), demonstrating promising generalizability despite institutional, temporal, and patient demographic differences.
Conclusions
This study presents one of the first triage ML models externally validated on a distinct ED cohort, achieving a new benchmark for AUPRC in flagging critically ill patients within minutes. Future directions include multi-site training and validation to further enhance real-world generalizability and clinical applicability.
