Enhancing Early Warning Outbreak Detection Using Multi Model Stacking Ensemble
Discuss this preprint
Start a discussion What are Sciety discussions?Listed in
This article is not in any list yet, why not save it to one of your lists.Abstract
Evaluating outbreak detection models is a key component of syndromic surveillance. However, balancing timeliness, predictive performance, and local surveillance constraints remains a major challenge. We developed and assessed whether stacking ensemble approaches, which integrate multiple outbreak detection methods, can improve the timeliness and predictive performance of influenza-like illness (ILI) surge detection.
Methods
We developed a two-stage stacking ensemble framework to detect early warning of ILI surges in city-level Primary Health Care encounter time series from Brazil (2022–2025). Epidemic thresholds were defined using the Moving Epidemic Method (MEM). In the first stage, multiple outbreak detection models (ODMs) generated warnings of unusual ILI activity. In the second, these warnings were then used as inputs to three supervised meta-classifiers: Logistic Regression, Extreme Gradient Boosting (XGB), and a Multi-layer Perceptron (MLP). For comparison, a Majority Voting (MV) aggregation is also examined. Timeliness, sensitivity, specificity, positive and negative predictive values are evaluated to measure each model’s ability to anticipate epidemic periods of varying intensity in 2025. Robustness was further assessed using simulated outbreak scenarios with varying magnitudes and durations.
Findings
We identified 5,765 ILI surge onsets across 5,365 Brazilian municipalities in 2025. Compared with individual ODMs and MV, stacking ensemble meta-classifiers anticipated up to 33% of surge onsets three weeks in advance (an average improvement of 15 percentage points) while reducing missed detections to <10%. They achieved sensitivity >90%, while maintaining balanced specificity >80%, PPV >65%, and NPV >99%. Improvements were greatest for very high-intensity surges, with missed detections reduced by more than half compared with individual ODMs. In simulated outbreak scenarios, the MLP and XGB classifiers remained robust despite being trained on fewer than half of all simulated surge events, consistently outperforming individual detection methods and simpler integration approaches.
Interpretation
We provide a practical framework for integrating complementary ODMs into a single, robust early warning decision. By improving both timeliness and predictive performance without requiring additional surveillance data or resources, this approach offers a scalable methodological upgrade for syndromic surveillance systems and supports more reliable public health decision-making.
Funding
The Rockefeller Foundation (award 2023 PPI 007 to MB-N); Brazilian National Research Council (CNPq (408775/2024-6); MB-N, PIPR, RFSA are CNPq fellows.
Research in context
Evidence before this study
We searched PubMed up to June 2025 without language or date restrictions to identify studies evaluating outbreak detection methods (ODMs) for syndromic surveillance (using the terms (“early warning system” OR “syndromic surveillance” OR “outbreak detection“) AND (“infectious disease*” OR “communicable disease*“) AND (“timeliness” OR “sensitivity” OR “specificity” OR “predictive value“)) and studies combining multiple ODMs (using (“outbreak detection model*” OR “aberration detection“) AND (“ensemble” OR “stacking” OR “meta-classifier” OR “model combination“) AND surveillance). Of 458 records screened, 45 were relevant. Most studies (32) compared individual ODMs using simulated outbreaks over synthetic surveillance baselines, reporting sensitivity (approximately 48–99%), specificity, false-alarm rate, and timeliness (hours to a few days). Although ensemble methods are widely used in biomedical machine learning, their application to outbreak detection has been limited. Existing infectious disease applications primarily combine forecasting models or construct reference labels to evaluate early warning systems, rather than integrating first-stage ODM outputs through supervised learning. Only five studies combined multiple ODMs for a single detection decision, and all relied on simple aggregation rules or expert review rather than learned meta-classifiers. The remaining studies (8) are narrative reviews or do not include quantitative data. They particularly point out the absence of standardised evaluation frameworks for syndromic surveillance systems and the need for methods that reflect the heterogeneity of local surveillance contexts and data constraints.
Added value of this study
To our knowledge, this is the first study to evaluate a two-stage stacking ensemble in which warnings from multiple independent ODMs serve as input features for supervised meta-classifiers to detect ILI surges using real-world, city-level primary health care data. Unlike prior combination approaches, our framework learns how to weight and integrate first-stage ODM outputs, reports standard surveillance performance metrics, and reflects a representative balance of modelling techniques adaptable to different data constraints and local contexts. We also evaluate the approach using simulated outbreaks superimposed on real municipal surveillance time series, bridging the gap between synthetic benchmarking and operational surveillance.
Implications of all the available evidence
Current syndromic surveillance systems based on individual ODMs or simple combination rules often show moderate performances. Our findings suggest that a stacking ensemble framework can meaningfully improve both timeliness and predictive performance of outbreak detection without requiring additional surveillance infrastructure. This offers local health authorities a low-cost methodological upgrade to existing syndromic surveillance pipelines and supports the integration of diverse modelling approaches within routine public health practice.
