Short-term forecasts to inform the response to the Covid-19 epidemic in the UK

  1. S Funk
  2. S Abbott
  3. BD Atkins
  4. M Baguelin
  5. JK Baillie
  6. P Birrell
  7. J Blake
  8. NI Bosse
  9. J Burton
  10. J Carruthers
  11. NG Davies
  12. D De Angelis
  13. L Dyson
  14. WJ Edmunds
  15. RM Eggo
  16. NM Ferguson
  17. K Gaythorpe
  18. E Gorsich
  19. G Guyver-Fletcher
  20. J Hellewell
  21. EM Hill
  22. A Holmes
  23. TA House
  24. C Jewell
  25. M Jit
  26. T Jombart
  27. I Joshi
  28. MJ Keeling
  29. E Kendall
  30. ES Knock
  31. AJ Kucharski
  32. KA Lythgoe
  33. SR Meakin
  34. JD Munday
  35. PJM Openshaw
  36. CE Overton
  37. F Pagani
  38. J Pearson
  39. PN Perez-Guzman
  40. L Pellis
  41. F Scarabel
  42. MG Semple
  43. K Sherratt
  44. M Tang
  45. MJ Tildesley
  46. E Van Leeuwen
  47. LK Whittles
  48. CMMID COVID-19 Working Group
  49. Imperial College COVID-19 Response Team
  50. ISARIC4C Investigators

Reviewed by

Read the full article See related articles

Listed in

Log in to save this article

Abstract

Background

Short-term forecasts of infectious disease can aid situational awareness and planning for outbreak response. Here, we report on multi-model forecasts of Covid-19 in the UK that were generated at regular intervals starting at the end of March 2020, in order to monitor expected healthcare utilisation and population impacts in real time.

Methods

We evaluated the performance of individual model forecasts generated between 24 March and 14 July 2020, using a variety of metrics including the weighted interval score as well as metrics that assess the calibration, sharpness, bias and absolute error of forecasts separately. We further combined the predictions from individual models into ensemble forecasts using a simple mean as well as a quantile regression average that aimed to maximise performance. We compared model performance to a null model of no change.

Results

In most cases, individual models performed better than the null model, and ensembles models were well calibrated and performed comparatively to the best individual models. The quantile regression average did not noticeably outperform the mean ensemble.

Conclusions

Ensembles of multi-model forecasts can inform the policy response to the Covid-19 pandemic by assessing future resource needs and expected population impact of morbidity and mortality.

Article activity feed

  1. Version published to 10.1101/2020.11.11.20220962v2 on medRxiv
  2. ScreenIT

    SciScore for 10.1101/2020.11.11.20220962: (What is this?)

    Please note, not all rigor criteria are appropriate for all manuscripts.

    Table 1: Rigor

    NIH rigor criteria are not applicable to paper type.

    Table 2: Resources

    No key resources detected.

    Results from OddPub: Thank you for sharing your code and data.

    Results from LimitationRecognizer: An explicit section about the limitations of the techniques employed in this study was not found. We encourage authors to address study limitations.

    Results from TrialIdentifier: No clinical trial numbers were referenced.

    Results from Barzooka: We did not find any issues relating to the usage of bar graphs.

    Results from JetFighter: We did not find any issues relating to colormaps.

    Results from rtransparent:
    • Thank you for including a conflict of interest statement. Authors are encouraged to include this statement when submitting to a journal.
    • Thank you for including a funding statement. Authors are encouraged to include this statement when submitting to a journal.
    • No protocol registration statement was detected.

    About SciScore

    SciScore is an automated tool that is designed to assist expert reviewers by finding and presenting formulaic information scattered throughout a paper in a standard, easy to digest format. SciScore checks for the presence and correctness of RRIDs (research resource identifiers), and for rigor criteria such as sex and investigator blinding. For details on the theoretical underpinning of rigor criteria and the tools shown here, including references cited, please follow this link.

  3. Version published to 10.1101/2020.11.11.20220962v1 on medRxiv