Hospital-wide, dynamic, individualized prediction of central line-associated bloodstream infections - development and temporal evaluation of six prediction models

Background

Central line-associated bloodstream infections (CLABSI) are preventable hospital-acquired infections. Predicting CLABSI helps improve early intervention strategies and enhance patient safety.

Aim

To develop and temporally evaluate dynamic prediction models for continuous CLABSI risk monitoring.

Methods

Data from hospitalized patients with central catheter(s) admitted to University Hospitals Leuven between 2014 and 2017 were used to develop five dynamic models (a cause-specific landmark supermodel, two random forest models, and two XGBoost models) to predict 7-day CLABSI risk, accounting for competing events (death, discharge, and catheter removal). The models’ predictions were then combined using a superlearner model. All models were temporally evaluated on data from the same hospital from 2018 to 2020 using performance metrics for discrimination, calibration, and clinical utility.

Findings

Among 61629 catheter episodes in the training set, 1930 (3.1%) resulted in CLABSI, while in the test set of 44544 catheter episodes, 1059 (2.4%) experienced CLABSI.

Among individual models, one XGBoost model reached an AUROC of 0.748. Calibration was good for predicted risks up to 5%, while the cause-specific and XGBoost models overestimated higher predicted risks. The superlearner displayed a modest improvement in discrimination (AUROC up to 0.751) and better calibration than the cause-specific and XGBoost models, but worse than the random forest models. The models showed clinical utility to support standard care interventions (at risk thresholds between 0.5-4%), but not to support advanced interventions (at thresholds 15-25%). A deterioration in model performance over time was observed on temporal evaluation.

Conclusion

Hospital-wide CLABSI prediction models offer clinical utility, though temporal evaluation revealed dataset shift.