Performance of early warning signals for disease re-emergence: A case study on COVID-19 data

  1. Daniele Proverbio
  2. Françoise Kemp
  3. Stefano Magni
  4. Jorge Gonçalves

Reviewed by

Read the full article See related articles

Listed in

Log in to save this article

Abstract

Developing measures for rapid and early detection of disease re-emergence is important to perform science-based risk assessment of epidemic threats. In the past few years, several early warning signals (EWS) from complex systems theory have been introduced to detect impending critical transitions and extend the set of indicators. However, it is still debated whether they are generically applicable or potentially sensitive to some dynamical characteristics such as system noise and rates of approach to critical parameter values. Moreover, testing on empirical data has, so far, been limited. Hence, verifying EWS performance remains a challenge. In this study, we tackle this question by analyzing the performance of common EWS, such as increasing variance and autocorrelation, in detecting the emergence of COVID-19 outbreaks in various countries. Our work illustrates that these EWS might be successful in detecting disease emergence when some basic assumptions are satisfied: a slow forcing through the transitions and not-fat-tailed noise. In uncertain cases, we observe that noise properties or commensurable time scales may obscure the expected early warning signals. Overall, our results suggest that EWS can be useful for active monitoring of epidemic dynamics, but that their performance is sensitive to certain features of the underlying dynamics. Our findings thus pave a connection between theoretical and empirical studies, constituting a further step towards the application of EWS indicators for informing public health policies.

Article activity feed

  1. Version published to 10.1371/journal.pcbi.1009958
  2. Version published to 10.1101/2021.03.30.21254631v2 on medRxiv
  3. ScreenIT

    SciScore for 10.1101/2021.03.30.21254631: (What is this?)

    Please note, not all rigor criteria are appropriate for all manuscripts.

    Table 1: Rigor

    NIH rigor criteria are not applicable to paper type.

    Table 2: Resources

    Software and Algorithms
    SentencesResources
    For the fitting, we used the scipy Python library.
    Python
    suggested: (IPython, RRID:SCR_001658)
    All indicators were estimate with their corresponding Matlab functions.
    Matlab
    suggested: (MATLAB, RRID:SCR_001622)

    Results from OddPub: Thank you for sharing your code and data.

    Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:
    Finally, we analysed the limitations of the indicator system in other contexts, characterised by different dynamical features such as rapid increases of R(t). This remark that, for EWS to properly work, the real system must fulfill the discussed conditions underlying the EWS modelling. These highlighted open problems remark that knowledge of the type of collected data is imperative to avoid misleading judgements in response to time series trends: EWS as epidemiological constructs will only remain valuable and relevant when used and interpreted correctly [41]. Our results thus constitute a further step towards the validation of literature findings and call for future studies, which will contribute to the exciting field of EWS in epidemic control and will likely have a tremendous impact in informing public health policies.

    Results from TrialIdentifier: No clinical trial numbers were referenced.

    Results from Barzooka: We did not find any issues relating to the usage of bar graphs.

    Results from JetFighter: We did not find any issues relating to colormaps.

    Results from rtransparent:
    • Thank you for including a conflict of interest statement. Authors are encouraged to include this statement when submitting to a journal.
    • Thank you for including a funding statement. Authors are encouraged to include this statement when submitting to a journal.
    • No protocol registration statement was detected.

    About SciScore

    SciScore is an automated tool that is designed to assist expert reviewers by finding and presenting formulaic information scattered throughout a paper in a standard, easy to digest format. SciScore checks for the presence and correctness of RRIDs (research resource identifiers), and for rigor criteria such as sex and investigator blinding. For details on the theoretical underpinning of rigor criteria and the tools shown here, including references cited, please follow this link.

  4. Version published to 10.1101/2021.03.30.21254631v1 on medRxiv