External Validation, Re-Calibration, and Extension of a Prediction Model of Early Acute Kidney Injury in Critically Ill Children using Multi-Center Data
Listed in
This article is not in any list yet, why not save it to one of your lists.Abstract
Background
Acute kidney injury (AKI) is common among children with critical illness and is associated with high morbidity and mortality. Risk prediction models designed for clinical decision support implementation offer an opportunity to identify and proactively mitigate AKI risks. Existing models have been primarily validated on single-center data, owing partly to the lack of appropriately detailed multicenter datasets.
Objective
To determine the accuracy of a single-center model to predict new AKI at 72 hours of ICU admission across two multicenter datasets and extend this model to improve prediction accuracy while maintaining acceptable alert burden.
Derivation and Validation Cohorts
We separately derived models in two datasets: PEDSNET-VPS, created through the linkage of PEDSnet electronic health record (EHR) extraction with Virtual Pediatric Systems (VPS); and the PICU Data Collaborative dataset, created through EHR extraction and harmonization from eight participating institutions. Derivation datasets comprised temporal and location-specific spit of these datasets (80%), while the holdout test split comprised the remaining (20%).
Prediction Model
We recalibrated an existing single-center model and measured discrimination and accuracy. We then add features guided by precision and recall measures. All features were available at 12 hours of ICU admission. We measure discrimination and accuracy at multiple cut-points and identify the features contributing most to the risk score.
Results
In two datasets comprising 186,540 ICU admissions, we report an incidence of early AKI of 2.2 – 2.7%. Initial recalibration of an existing single-center model demonstrated poor discrimination (AUROC 0.60 – 0.78). Following the addition of new features, we report higher AUROC values of 0.79 - 0.80 and AUPRC values of 0.13 – 0.21 in both datasets. We report accuracy at several cutpoints as well as cross-validate between datasets.
Conclusions
In this first use of two new multicenter datasets, we report improved discrimination and accuracy in a model designed specifically for implementation, balancing sensitivity and precision to predict patients at risk for AKI development.
