A Comparative Analysis of Statistical Methods to Estimate the Reproduction Number in Emerging Epidemics, With Implications for the Current Coronavirus Disease 2019 (COVID-19) Pandemic

  1. Megan O’Driscoll
  2. Carole Harry
  3. Christl A Donnelly
  4. Anne Cori
  5. Ilaria Dorigatti

Reviewed by

Read the full article

Listed in

Log in to save this article

Abstract

Background

As the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) pandemic continues its rapid global spread, quantification of local transmission patterns has been, and will continue to be, critical for guiding the pandemic response. Understanding the accuracy and limitations of statistical methods to estimate the basic reproduction number, R0, in the context of emerging epidemics is therefore vital to ensure appropriate interpretation of results and the subsequent implications for control efforts.

Methods

Using simulated epidemic data, we assess the performance of 7 commonly used statistical methods to estimate R0 as they would be applied in a real-time outbreak analysis scenario: fitting to an increasing number of data points over time and with varying levels of random noise in the data. Method comparison was also conducted on empirical outbreak data, using Zika surveillance data from the 2015–2016 epidemic in Latin America and the Caribbean.

Results

We find that most methods considered here frequently overestimate R0 in the early stages of epidemic growth on simulated data, the magnitude of which decreases when fitted to an increasing number of time points. This trend of decreasing bias over time can easily lead to incorrect conclusions about the course of the epidemic or the need for control efforts.

Conclusions

We show that true changes in pathogen transmissibility can be difficult to disentangle from changes in methodological accuracy and precision in the early stages of epidemic growth, particularly for data with significant over-dispersion. As localized epidemics of SARS-CoV-2 take hold around the globe, awareness of this trend will be important for appropriately cautious interpretation of results and subsequent guidance for control efforts.

Article activity feed

  1. Version published to 10.1093/cid/ciaa1599
  2. ScreenIT

    SciScore for 10.1101/2020.05.13.20101121: (What is this?)

    Please note, not all rigor criteria are appropriate for all manuscripts.

    Table 1: Rigor

    NIH rigor criteria are not applicable to paper type.

    Table 2: Resources

    No key resources detected.

    Results from OddPub: Thank you for sharing your code and data.

    Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:
    We observed varying strengths and limitations associated with each of the methods considered here and the sensitivity of these factors to levels of random noise and the amount of available data. While the linear exponential growth rate method performed poorest in terms of coverage of actual R0 values when fit to simulated data with no added noise, it had the highest coverage of actual R0 values when fit to data with added random noise, driven by increased confidence interval widths in the presence of overdispersion. Conversely, the Wallinga and Teunis method had the highest coverage of actual R0 values when fitted to data with no noise and mild (Poisson) noise but had the largest average bias across all time points when fitted to data with high overdispersion (negative binomial noise). There are a number of limitations in our analysis, including the simplicity inherent in the assumptions of homogenous mixing in fully susceptible and closed populations that were used to simulate the case data. These assumptions do not reproduce the sub-exponential growth dynamics that can occur in empirical epidemics as driven by complex spatial structures, local contact networks and other socio-behavioural factors not accounted for in this analysis (24, 25). The results of this analysis are based on weekly epidemic case-time series, which may not fully reflect results obtained when daily case reports are available. However, whilst the larger number of datapoints provided by daily time series ...

    Results from TrialIdentifier: No clinical trial numbers were referenced.

    Results from Barzooka: We did not find any issues relating to the usage of bar graphs.

    Results from JetFighter: We did not find any issues relating to colormaps.

    Results from rtransparent:
    • Thank you for including a conflict of interest statement. Authors are encouraged to include this statement when submitting to a journal.
    • Thank you for including a funding statement. Authors are encouraged to include this statement when submitting to a journal.
    • No protocol registration statement was detected.

    About SciScore

    SciScore is an automated tool that is designed to assist expert reviewers by finding and presenting formulaic information scattered throughout a paper in a standard, easy to digest format. SciScore checks for the presence and correctness of RRIDs (research resource identifiers), and for rigor criteria such as sex and investigator blinding. For details on the theoretical underpinning of rigor criteria and the tools shown here, including references cited, please follow this link.

  3. Version published to 10.1101/2020.05.13.20101121v1 on medRxiv