Can ensemble methods improve predictive performance of existing models estimating chronic kidney disease among patients with diabetes?

Clinical prediction models often suffer from poor model transportability and/or subgroup performance resulting from using a single data source. We aimed to determine whether ensemble methods can combine multiple existing models to improve predictive performance when compared to component models. As a case study, we used electronic medical records from the Canadian Primary Care Sentinel Surveillance Network (CPCSSN) to test ensemble methods for models estimating the risk of developing chronic kidney disease (CKD) among people with diabetes in a cohort of 37,604 individuals. We considered 13 models identified from prior systematic reviews and combined their unique risk estimates using many strategies (e.g., averaging or mixture-of-experts). We assessed discrimination, precision, recall, calibration, net reclassification index, and integrated discrimination improvement. Ensemble methods performed well, but no better than the best performing component model. This study suggests ensemble methods may not improve predictive performance, though further research should confirm these findings.