Projected resurgence of COVID-19 in the United States in July—December 2021 resulting from the increased transmissibility of the Delta variant and faltering vaccination
Curation statements for this article:-
Curated by eLife
Evaluation Summary:
In this paper, the authors presented the joint efforts from nine modeling teams to provide six-month projection of the COVID-19 pandemic across the US, in view of the circulation of the more transmissible Delta variant. All of the nine models projected substantial Fall resurgences based on data as of 3 July 2021, but the actual resurgence scale as of 31 July 2021 had exceeded the projections of all of the nine models. This suggests that transmission may be even higher than expected given model assumptions, and that forecasts beyond more than a few weeks are likely to be highly uncertain. This paper will be of high interest to public health specialists, forecast modelers, and members of the general public interested in the evolution of the COVID-19 pandemic and the impact of public health interventions in the USA.
(This preprint has been reviewed by eLife. We include the public reviews from the reviewers here; the authors also receive private feedback with suggested changes to the manuscript. Reviewer #3 agreed to share their name with the authors.)
This article has been Reviewed by the following groups
Listed in
- Evaluated articles (eLife)
- Evaluated articles (ScreenIT)
Abstract
In Spring 2021, the highly transmissible SARS-CoV-2 Delta variant began to cause increases in cases, hospitalizations, and deaths in parts of the United States. At the time, with slowed vaccination uptake, this novel variant was expected to increase the risk of pandemic resurgence in the US in summer and fall 2021. As part of the COVID-19 Scenario Modeling Hub, an ensemble of nine mechanistic models produced 6-month scenario projections for July–December 2021 for the United States. These projections estimated substantial resurgences of COVID-19 across the US resulting from the more transmissible Delta variant, projected to occur across most of the US, coinciding with school and business reopening. The scenarios revealed that reaching higher vaccine coverage in July–December 2021 reduced the size and duration of the projected resurgence substantially, with the expected impacts was largely concentrated in a subset of states with lower vaccination coverage. Despite accurate projection of COVID-19 surges occurring and timing, the magnitude was substantially underestimated 2021 by the models compared with the of the reported cases, hospitalizations, and deaths occurring during July–December, highlighting the continued challenges to predict the evolving COVID-19 pandemic. Vaccination uptake remains critical to limiting transmission and disease, particularly in states with lower vaccination coverage. Higher vaccination goals at the onset of the surge of the new variant were estimated to avert over 1.5 million cases and 21,000 deaths, although may have had even greater impacts, considering the underestimated resurgence magnitude from the model.
Article activity feed
-
-
Evaluation Summary:
In this paper, the authors presented the joint efforts from nine modeling teams to provide six-month projection of the COVID-19 pandemic across the US, in view of the circulation of the more transmissible Delta variant. All of the nine models projected substantial Fall resurgences based on data as of 3 July 2021, but the actual resurgence scale as of 31 July 2021 had exceeded the projections of all of the nine models. This suggests that transmission may be even higher than expected given model assumptions, and that forecasts beyond more than a few weeks are likely to be highly uncertain. This paper will be of high interest to public health specialists, forecast modelers, and members of the general public interested in the evolution of the COVID-19 pandemic and the impact of public health interventions in the USA.
(Thi…
Evaluation Summary:
In this paper, the authors presented the joint efforts from nine modeling teams to provide six-month projection of the COVID-19 pandemic across the US, in view of the circulation of the more transmissible Delta variant. All of the nine models projected substantial Fall resurgences based on data as of 3 July 2021, but the actual resurgence scale as of 31 July 2021 had exceeded the projections of all of the nine models. This suggests that transmission may be even higher than expected given model assumptions, and that forecasts beyond more than a few weeks are likely to be highly uncertain. This paper will be of high interest to public health specialists, forecast modelers, and members of the general public interested in the evolution of the COVID-19 pandemic and the impact of public health interventions in the USA.
(This preprint has been reviewed by eLife. We include the public reviews from the reviewers here; the authors also receive private feedback with suggested changes to the manuscript. Reviewer #3 agreed to share their name with the authors.)
-
Reviewer #1 (Public Review):
This manuscript presents predictions of COVID-19 cases, hospitalizations, and deaths over July-December 2021 in the US using data up to July 2021, and combining the predictions from nine different models. The predicted resurgence in cases during late summer has been largely borne out by the data that has come out since. The results have some important implications for public health policy, as discussed below.
Unsurprisingly, the models find that outcomes over the next six months are highly dependent on prior and future vaccination coverage by state, reinforcing the importance of increasing vaccination coverage. The authors also theorize some of the variation in model predictions may be due to non-pharmaceutical intervention adherence, however they do not explore this relationship in this paper. It would have …
Reviewer #1 (Public Review):
This manuscript presents predictions of COVID-19 cases, hospitalizations, and deaths over July-December 2021 in the US using data up to July 2021, and combining the predictions from nine different models. The predicted resurgence in cases during late summer has been largely borne out by the data that has come out since. The results have some important implications for public health policy, as discussed below.
Unsurprisingly, the models find that outcomes over the next six months are highly dependent on prior and future vaccination coverage by state, reinforcing the importance of increasing vaccination coverage. The authors also theorize some of the variation in model predictions may be due to non-pharmaceutical intervention adherence, however they do not explore this relationship in this paper. It would have been interesting to see if there was also a correlation between the predicted number of cases per state and non-pharmaceutical intervention adherence across states in July, but this information is possibly more difficult to acquire.
The combination of predictions from different models is a strength, as it allows to assess future uncertainty related to different model structures and parameter assumptions. However, it is disappointing that this paper does not capitalize on this strength by also showing a comparison of individual model predictions, other than by showing the prediction intervals. The authors mentioned the models varied both in the magnitude of COVID-19 incidence, but also in the timing of the peak of infection. It would have been interesting to see a figure showing a cross-model comparison of predictions for a given scenario (the most suitable scenario given the data would have likely been the worst-case low vaccination-high variant transmissibility scenario). This would have allowed a better understanding of uncertainty related to different model assumptions, and would have allowed the reader to assess whether the median captures well the central tendency across models.An interesting feature of this paper is the comparison of model predictions against prospective data which was not used to fit the model. This comparison shows that reported cases up to July 31 exceeded those projected in all scenarios. This discrepancy should not be interpreted as indicating the models' predictions are invalid, but rather as a healthy and important exercise in reassessing our assumptions regarding the spread of SARS-CoV-2. It suggests that current SARS-CoV-2 variants in the US might for example be more transmissible than what was assumed in the models, or that adherence to non-pharmaceutical interventions is lower than anticipated. It would have been useful if the authors could have further explored which models with which assumptions were most closely able to match the earlier surge in cases. Nonetheless, even though the models may have underestimated the speed at which the cases would start to increase in July, there is a very strong correlation between the observed and predicted number of cases by state during July. This suggests that though the absolute number of COVID-19 cases and deaths the models predict over the next 6 months could be be an underestimate, the models do capture the major drivers of epidemic surges and are able to predict with good accuracy which states are likely to be most impacted by the pandemic over the next few months.
-
Reviewer #2 (Public Review):
The paper is well written, and data and codes have been made publicly available.
The Delta wave in the US has been descending since early September. So it looks like two scenarios (i.e., high/low vaccination + high variant transmissibility) in Figure 1 predicted the timing of the decline quite well.
-
Reviewer #3 (Public Review):
This paper reports on rounds 6 and 7 of the COVID-19 scenario modelling hub projections for the US COVID-19 epidemic. The specific focus of this round is the impact of the more transmissible Delta variant during the second half of 2021, for a range of vaccine uptake and virus transmissibility assumptions.
The ensemble models show logical relative differences in the likely magnitude of impacts of COVID over the period of interest given the various paired assumptions. The uncertainty in these estimates is very high and observed data following the period of projection up the time of submission are tracking along the 95% prediction interval.
Looking at what's happened since, my sense is that the models assuming high transmissibility have captured the time course reasonably well but vastly underestimated …
Reviewer #3 (Public Review):
This paper reports on rounds 6 and 7 of the COVID-19 scenario modelling hub projections for the US COVID-19 epidemic. The specific focus of this round is the impact of the more transmissible Delta variant during the second half of 2021, for a range of vaccine uptake and virus transmissibility assumptions.
The ensemble models show logical relative differences in the likely magnitude of impacts of COVID over the period of interest given the various paired assumptions. The uncertainty in these estimates is very high and observed data following the period of projection up the time of submission are tracking along the 95% prediction interval.
Looking at what's happened since, my sense is that the models assuming high transmissibility have captured the time course reasonably well but vastly underestimated magnitude. Given all of the very reasonably acknowledged uncertainties, it's not surprising that projections beyond a few weeks are rarely informative, but the conclusions regarding the suitability of this approach to inform response should be tempered. Forecasts beyond more than a few weeks are unlikely to be sufficiently valid for public health and clinical preparedness, which these findings clearly show.
The ensemble modelling exercise is very useful, but there's a lot of information lost in merging the outputs of models that make different critical assumptions about VE against infectiousness and age distribution of uptake, which were allowed to vary between models, with other assumptions. I would be very interested to hear more about the variation in outputs that went into the ensemble and which aggregate assumptions have resulted in trajectories that have been more accurately predictive of the observed epidemic. This process is important for validation and improvement of future rounds.
At the level of states there is clear heterogeneity in coverage and mandated or spontaneous behaviours that drive variation in outcomes. The models seem to capture those rankings well, even if the absolute values prove to be underestimates. It would be interesting to understand how heterogeneity below the level of states contributes to the overestimation of vaccine impact and whether this is also more marked in some states than others.
There's no mention of waning immunity but given the time at which the US rollout commenced, one anticipates declining effectiveness against transmission in this observation window for early uptake cohorts.
-
SciScore for 10.1101/2021.08.28.21262748: (What is this?)
Please note, not all rigor criteria are appropriate for all manuscripts.
Table 1: Rigor
NIH rigor criteria are not applicable to paper type.Table 2: Resources
No key resources detected.
Results from OddPub: Thank you for sharing your code and data.
Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:The findings in this report are subject to several limitations. First, considerable uncertainty is inherent to long-term projections. This has been repeatedly illustrated throughout the COVID-19 pandemic, with rapid changes in behavior, deployment of vaccines and boosters, and the emergence of novel variants, each of which has the capacity to drastically shift the epidemic trajectories. Uncertainty may arise from three main sources: …
SciScore for 10.1101/2021.08.28.21262748: (What is this?)
Please note, not all rigor criteria are appropriate for all manuscripts.
Table 1: Rigor
NIH rigor criteria are not applicable to paper type.Table 2: Resources
No key resources detected.
Results from OddPub: Thank you for sharing your code and data.
Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:The findings in this report are subject to several limitations. First, considerable uncertainty is inherent to long-term projections. This has been repeatedly illustrated throughout the COVID-19 pandemic, with rapid changes in behavior, deployment of vaccines and boosters, and the emergence of novel variants, each of which has the capacity to drastically shift the epidemic trajectories. Uncertainty may arise from three main sources: specification of the scenarios (e.g., uncertainty in transmissibility); errors in the structure or assumptions of individual models given a specific scenario (e.g., variations in assumptions about vaccination uptake); and inaccurate calibration based on incomplete or biased data (e.g., reporting backlogs). None of the 4 scenarios considered here are likely to precisely reflect the future reality over a 6-month period.2 Further, for a given scenario, there is notable variation among individual model projections with regards to both the timing and the magnitude of the resurgence. Variation likely reflects differences in model structure, projected vaccine coverage, projected variant growth, and importance of seasonal effects. In addition, these scenarios do not specify considerations of Delta infecting previously immune individuals, the waning of existing immunity, increases in NPIs, or vaccination among children aged <12 years, all of which may be important drivers of dynamics in the coming months. In the same vein, model estimates are dependent on ...
Results from TrialIdentifier: No clinical trial numbers were referenced.
Results from Barzooka: We did not find any issues relating to the usage of bar graphs.
Results from JetFighter: We did not find any issues relating to colormaps.
Results from rtransparent:- Thank you for including a conflict of interest statement. Authors are encouraged to include this statement when submitting to a journal.
- Thank you for including a funding statement. Authors are encouraged to include this statement when submitting to a journal.
- No protocol registration statement was detected.
Results from scite Reference Check: We found no unreliable references.
-
-