FOCUS: Forecasting COVID-19 in the United States
This article has been Reviewed by the following groups
Listed in
- Evaluated articles (ScreenIT)
Abstract
Infectious disease forecasting has been a useful tool for public health planning and messaging during the COVID-19 pandemic. In partnership with the CDC, the organizers of the COVID-19 Forecast Hub have created a mechanism for forecasters from academia, industry, and government organizations to submit weekly near-term predictions of COVID-19 targets in the United States. Here we describe our efforts to participate in the COVID-19 Forecast Hub through the Fo recasting CO VID-19 in the U nited S tates ( FOCUS ) project. The effort led to more than three months of weekly submissions and development of an automated pipeline to generate forecasts. The models used in FOCUS yielded forecasts that ranked relatively well in terms of precision and accuracy.
Article activity feed
-
SciScore for 10.1101/2021.05.18.21257386: (What is this?)
Please note, not all rigor criteria are appropriate for all manuscripts.
Table 1: Rigor
NIH rigor criteria are not applicable to paper type.Table 2: Resources
No key resources detected.
Results from OddPub: Thank you for sharing your code and data.
Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:The FOCUS project had notable limitations. The SigSci-TS models used time series approaches. Autoregressive models have been shown to be successful in flu forecasting, and forecasts from our time series models generally appeared to score well for the period evaluated. However, we acknowledge that at different points in the pandemic (e.g. early on when there is very little data collected) a time series model may not be sufficient. …
SciScore for 10.1101/2021.05.18.21257386: (What is this?)
Please note, not all rigor criteria are appropriate for all manuscripts.
Table 1: Rigor
NIH rigor criteria are not applicable to paper type.Table 2: Resources
No key resources detected.
Results from OddPub: Thank you for sharing your code and data.
Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:The FOCUS project had notable limitations. The SigSci-TS models used time series approaches. Autoregressive models have been shown to be successful in flu forecasting, and forecasts from our time series models generally appeared to score well for the period evaluated. However, we acknowledge that at different points in the pandemic (e.g. early on when there is very little data collected) a time series model may not be sufficient. Furthermore, while the SigSci-TS forecasts performed well for the near-term (1 to 4 week) horizons, we cannot and should not extrapolate performance evaluation to longer horizons or scenario projections. If applied to longer horizons, the SigSci-TS approach would have no way to bound predictions at the size of the susceptible population. While we only developed forecast submissions using time series methods, we did cursorily explore other approaches including compartmental models. Future work may involve development of a composite modeling approach calibrated to horizon. Such a framework might weight a time series model differently than another method (e.g. a compartmental model) given the number of weeks ahead for the forecast. In theory this could leverage the strengths of each constituent approach while mitigating model-specific limitations. Beyond limitations specific to the SigSci-TS approach, there are general challenges to forecasting COVID-19 outcomes in the United States that are worth noting. In particular, data reporting standards have not...
Results from TrialIdentifier: No clinical trial numbers were referenced.
Results from Barzooka: We did not find any issues relating to the usage of bar graphs.
Results from JetFighter: We did not find any issues relating to colormaps.
Results from rtransparent:- Thank you for including a conflict of interest statement. Authors are encouraged to include this statement when submitting to a journal.
- Thank you for including a funding statement. Authors are encouraged to include this statement when submitting to a journal.
- No protocol registration statement was detected.
Results from scite Reference Check: We found no unreliable references.
-
