The methodologies to assess the effectiveness of non-pharmaceutical interventions during COVID-19: a systematic review

  1. Nicolas Banholzer
  2. Adrian Lison
  3. Dennis Özcelik
  4. Tanja Stadler
  5. Stefan Feuerriegel
  6. Werner Vach

Reviewed by

Read the full article

Listed in

Log in to save this article

Abstract

Non-pharmaceutical interventions, such as school closures and stay-at-home orders, have been implemented around the world to control the spread of SARS-CoV-2. Their effectiveness in improving health-related outcomes has been the subject of numerous empirical studies. However, these studies show fairly large variation among methodologies in use, reflecting the absence of an established methodological framework. On the one hand, variation in methodologies may be desirable to assess the robustness of results; on the other hand, a lack of common standards can impede comparability among studies. To establish a comprehensive overview over the methodologies in use, we conducted a systematic review of studies assessing the effectiveness of non-pharmaceutical interventions between January 1, 2020 and January 12, 2021 (n = 248). We identified substantial variation in methodologies with respect to study setting, outcome, intervention, methodological approach, and effectiveness assessment. On this basis, we point to shortcomings of existing studies and make recommendations for the design of future studies.

Article activity feed

  1. Version published to 10.1007/s10654-022-00908-y
  2. ScreenIT

    SciScore for 10.1101/2022.04.14.22273858: (What is this?)

    Please note, not all rigor criteria are appropriate for all manuscripts.

    Table 1: Rigor

    NIH rigor criteria are not applicable to paper type.

    Table 2: Resources

    Software and Algorithms
    SentencesResources
    We searched the databases Embase, PubMed, Scopus, and Web of Science.
    Embase
    suggested: (EMBASE, RRID:SCR_001650)
    PubMed
    suggested: (PubMed, RRID:SCR_004846)
    These databases include, among others, MEDLINE, Biological Abstracts, CAB Abstracts, and Global Health.
    MEDLINE
    suggested: (MEDLINE, RRID:SCR_002185)

    Results from OddPub: Thank you for sharing your code and data.

    Results from LimitationRecognizer: An explicit section about the limitations of the techniques employed in this study was not found. We encourage authors to address study limitations.

    Results from TrialIdentifier: No clinical trial numbers were referenced.

    Results from Barzooka: We did not find any issues relating to the usage of bar graphs.

    Results from JetFighter: We did not find any issues relating to colormaps.

    Results from rtransparent:
    • Thank you for including a conflict of interest statement. Authors are encouraged to include this statement when submitting to a journal.
    • No funding statement was detected.
    • No protocol registration statement was detected.

    Results from scite Reference Check: We found no unreliable references.

    About SciScore

    SciScore is an automated tool that is designed to assist expert reviewers by finding and presenting formulaic information scattered throughout a paper in a standard, easy to digest format. SciScore checks for the presence and correctness of RRIDs (research resource identifiers), and for rigor criteria such as sex and investigator blinding. For details on the theoretical underpinning of rigor criteria and the tools shown here, including references cited, please follow this link.

  3. Version published to 10.1101/2022.04.14.22273858v1 on medRxiv