Population-scale machine learning models to estimate individual-level risk for acute respiratory infection
Listed in
This article is not in any list yet, why not save it to one of your lists.Abstract
Background
Models based on administrative population health records can help improve understanding of disease and hospitalisation risk. This understanding can support evidence-based population health policy and processes. Influenza and other acute respiratory diseases place a large, inequitable burden on the New Zealand population and health care resources. We have developed and evaluated models using retrospective New Zealand hospitalisation records linked to other administrative data.
Methods
We analysed linked annual records for approximately 4·85–5·36 million people in 2017–2024. For each year, models were trained to distinguish people who experienced at least one hospitalisation associated with respiratory infectious disease from those without such a hospitalisation during that year. Six algorithms and eight variable sets were assessed using five-fold cross-validation grouped by demographic profile. Variables comprised demographics, continuous hospital-based (M3) and pharmaceutical-based (P3) morbidity indices, and influenza vaccination status. Models trained on one year were also evaluated on data from the following year. Performance measures were the area under the receiver operating characteristic curve (AUROC) and the average precision score (APS). SHapley Additive exPlanations (SHAP) described model contributions, including paired vaccination-input changes in 2024.
Findings
Annual national hospitalisation prevalence ranged from 1·01% to 1·60%. With all variables, mean cross-validation AUROC and APS were 0·8430 and 0·1283 for LightGBM (LGB), and 0·8415 and 0·1285 for XGBoost (XGB). Adding both morbidity indices to the demographics increased LGB APS from 0·0720 to 0·1158; adding vaccination further increased it to 0·1283. Next-year AUROC and APS averaged 0·8429 and 0·1267 for LGB and 0·8423 and 0·1277 for XGB. Age and morbidity were the most influential variables. Among vaccinated individuals, SHAP analysis indicated that vaccination contributed most strongly towards lower model-estimated hospitalisation risk in the oldest age group.
Interpretation
Combining historical morbidity and vaccination information in gradient boosting models improved the ability to distinguish between people who experienced respiratory hospitalisation and those who did not in annual New Zealand population data. Similar next-year performance and interpretable variation across age and morbidity profiles support further development toward practical risk assessment, including calibration and evaluation across population groups.
Research in context
Evidence before this study
Our focused narrative review considered population-based respiratory hospitalisation models, respiratory disease burden in New Zealand, and the development of the M3 and P3 morbidity indices. Existing studies include population models for COVID-19 and infant respiratory syncytial virus, risk-factor analyses of broad respiratory admissions, and evidence that hospital and pharmaceutical morbidity measures provide complementary information.
Added value of this study
This is the first evaluation of national hospitalisation data linked to other national administrative data to model the risk of acute respiratory infection admission across eight annual datasets, comparing six algorithms and eight variable sets. Together with the benchmark results, these findings illustrate how gradient boosting applied to population-scale records can represent and explain combinations of routinely recorded health information.
Implications of all the available evidence
Quantifying historical morbidity and including vaccination information improved respiratory hospitalisation classification, with gradient boosting providing the strongest overall performance. The explanatory analyses further showed that vaccination-related contributions depended on the individual’s age and morbidity profile. Representing these clinically plausible patterns supports the use of flexible, non-linear models, such as the tree-based algorithms, and rich population-scale records for developing interpretable respiratory risk assessments. The study establishes a basis for a New Zealand respiratory risk model by identifying effective algorithms and useful combinations of variables.
