Theoretical Framework for Retrospective Studies of the Effectiveness of SARS-CoV-2 Vaccines

  1. Joseph A. Lewnard
  2. Manish M. Patel
  3. Nicholas P. Jewell
  4. Jennifer R. Verani
  5. Miwako Kobayashi
  6. Mark W. Tenforde
  7. Natalie E. Dean
  8. Benjamin J. Cowling
  9. Benjamin A. Lopman

Reviewed by

Read the full article See related articles

Listed in

Log in to save this article

Abstract

Observational studies of the effectiveness of vaccines to prevent COVID-19 are needed to inform real-world use. Such studies are now underway amid the ongoing rollout of SARS-CoV-2 vaccines globally. Although traditional case-control and test-negative design studies feature prominently among strategies used to assess vaccine effectiveness, such studies may encounter important threats to validity. Here, we review the theoretical basis for estimation of vaccine direct effects under traditional case-control and test-negative design frameworks, addressing specific natural history parameters of SARS-CoV-2 infection and COVID-19 relevant to these designs. Bias may be introduced by misclassification of cases and controls, particularly when clinical case criteria include common, nonspecific indicators of COVID-19. When using diagnostic assays with high analytical sensitivity for SARS-CoV-2 detection, individuals testing positive may be counted as cases even if their symptoms are due to other causes. The traditional case-control design may be particularly prone to confounding due to associations of vaccination with healthcare-seeking behavior or risk of infection. The test-negative design reduces but may not eliminate this confounding, for instance, if individuals who receive vaccination seek care or testing for less-severe illness. These circumstances indicate the two study designs cannot be applied naively to datasets gathered through public health surveillance or administrative sources. We suggest practical strategies to reduce bias in vaccine effectiveness estimates at the study design and analysis stages.

Article activity feed

  1. Version published to 10.1097/ede.0000000000001366
  2. ScreenIT

    SciScore for 10.1101/2021.01.21.21250258: (What is this?)

    Please note, not all rigor criteria are appropriate for all manuscripts.

    Table 1: Rigor

    Institutional Review Board Statementnot detected.
    Randomizationnot detected.
    Blindingnot detected.
    Power Analysisnot detected.
    Sex as a biological variablenot detected.

    Table 2: Resources

    No key resources detected.

    Results from OddPub: We did not detect open data. We also did not detect open code. Researchers are encouraged to share open data when possible (see Nature blog).

    Results from LimitationRecognizer: An explicit section about the limitations of the techniques employed in this study was not found. We encourage authors to address study limitations.

    Results from TrialIdentifier: No clinical trial numbers were referenced.

    Results from Barzooka: We did not find any issues relating to the usage of bar graphs.

    Results from JetFighter: We did not find any issues relating to colormaps.

    Results from rtransparent:
    • Thank you for including a conflict of interest statement. Authors are encouraged to include this statement when submitting to a journal.
    • Thank you for including a funding statement. Authors are encouraged to include this statement when submitting to a journal.
    • No protocol registration statement was detected.

    About SciScore

    SciScore is an automated tool that is designed to assist expert reviewers by finding and presenting formulaic information scattered throughout a paper in a standard, easy to digest format. SciScore checks for the presence and correctness of RRIDs (research resource identifiers), and for rigor criteria such as sex and investigator blinding. For details on the theoretical underpinning of rigor criteria and the tools shown here, including references cited, please follow this link.

  3. Version published to 10.1101/2021.01.21.21250258v1 on medRxiv