National and subnational short-term forecasting of COVID-19 in Germany and Poland during early 2021

  1. Johannes Bracher
  2. Daniel Wolffram
  3. Jannik Deuschel
  4. Konstantin Görgen
  5. Jakob L. Ketterer
  6. Alexander Ullrich
  7. Sam Abbott
  8. Maria V. Barbarossa
  9. Dimitris Bertsimas
  10. Sangeeta Bhatia
  11. Marcin Bodych
  12. Nikos I. Bosse
  13. Jan Pablo Burgard
  14. Lauren Castro
  15. Geoffrey Fairchild
  16. Jochen Fiedler
  17. Jan Fuhrmann
  18. Sebastian Funk
  19. Anna Gambin
  20. Krzysztof Gogolewski
  21. Stefan Heyder
  22. Thomas Hotz
  23. Yuri Kheifetz
  24. Holger Kirsten
  25. Tyll Krueger
  26. Ekaterina Krymova
  27. Neele Leithäuser
  28. Michael L. Li
  29. Jan H. Meinke
  30. Błażej Miasojedow
  31. Isaac J. Michaud
  32. Jan Mohring
  33. Pierre Nouvellet
  34. Jedrzej M. Nowosielski
  35. Tomasz Ozanski
  36. Maciej Radwan
  37. Franciszek Rakowski
  38. Markus Scholz
  39. Saksham Soni
  40. Ajitesh Srivastava
  41. Tilmann Gneiting
  42. Melanie Schienle

Reviewed by

Read the full article See related articles

Listed in

Log in to save this article

Abstract

Background

During the COVID-19 pandemic there has been a strong interest in forecasts of the short-term development of epidemiological indicators to inform decision makers. In this study we evaluate probabilistic real-time predictions of confirmed cases and deaths from COVID-19 in Germany and Poland for the period from January through April 2021.

Methods

We evaluate probabilistic real-time predictions of confirmed cases and deaths from COVID-19 in Germany and Poland. These were issued by 15 different forecasting models, run by independent research teams. Moreover, we study the performance of combined ensemble forecasts. Evaluation of probabilistic forecasts is based on proper scoring rules, along with interval coverage proportions to assess calibration. The presented work is part of a pre-registered evaluation study.

Results

We find that many, though not all, models outperform a simple baseline model up to four weeks ahead for the considered targets. Ensemble methods show very good relative performance. The addressed time period is characterized by rather stable non-pharmaceutical interventions in both countries, making short-term predictions more straightforward than in previous periods. However, major trend changes in reported cases, like the rebound in cases due to the rise of the B.1.1.7 (Alpha) variant in March 2021, prove challenging to predict.

Conclusions

Multi-model approaches can help to improve the performance of epidemiological forecasts. However, while death numbers can be predicted with some success based on current case and hospitalization data, predictability of case numbers remains low beyond quite short time horizons. Additional data sources including sequencing and mobility data, which were not extensively used in the present study, may help to improve performance.

Article activity feed

  1. Version published to 10.1038/s43856-022-00191-8
  2. Version published to 10.1101/2021.11.05.21265810v2 on medRxiv
  3. ScreenIT

    SciScore for 10.1101/2021.11.05.21265810: (What is this?)

    Please note, not all rigor criteria are appropriate for all manuscripts.

    Table 1: Rigor

    NIH rigor criteria are not applicable to paper type.

    Table 2: Resources

    No key resources detected.

    Results from OddPub: Thank you for sharing your code and data.

    Results from LimitationRecognizer: An explicit section about the limitations of the techniques employed in this study was not found. We encourage authors to address study limitations.

    Results from TrialIdentifier: No clinical trial numbers were referenced.

    Results from Barzooka: We did not find any issues relating to the usage of bar graphs.

    Results from JetFighter: We did not find any issues relating to colormaps.

    Results from rtransparent:
    • Thank you for including a conflict of interest statement. Authors are encouraged to include this statement when submitting to a journal.
    • Thank you for including a funding statement. Authors are encouraged to include this statement when submitting to a journal.
    • No protocol registration statement was detected.

    Results from scite Reference Check: We found no unreliable references.

    About SciScore

    SciScore is an automated tool that is designed to assist expert reviewers by finding and presenting formulaic information scattered throughout a paper in a standard, easy to digest format. SciScore checks for the presence and correctness of RRIDs (research resource identifiers), and for rigor criteria such as sex and investigator blinding. For details on the theoretical underpinning of rigor criteria and the tools shown here, including references cited, please follow this link.

  4. Version published to 10.1101/2021.11.05.21265810v1 on medRxiv