Evaluating real-world COVID-19 vaccine effectiveness using a test-negative case–control design

This article has been Reviewed by the following groups

Read the full article See related articles

Abstract

Aim: It is important to assess if clinical trial efficacy translates into real-world effectiveness for COVID-19 vaccines. Materials & methods: We conducted a modified test-negative design (TND) to evaluate the real-world effectiveness of three COVID-19 vaccines. We defined cases in two ways: self-reported COVID-19-positive tests, and self-reported positive tests with ≥1 moderate/severe COVID-19 symptom. Results: Any vaccination was associated with a 95% reduction in subsequently reporting a positive COVID-19 test, and a 71% reduction in reporting a positive test and ≥1 moderate/severe symptom. Conclusion: We observed high effectiveness across all three marketed vaccines, both for self-reported positive COVID-19 tests and moderate/severe COVID-19 symptoms. This innovative TND approach can be implemented in future COVID-19 vaccine and treatment real-world effectiveness studies. Clinicaltrials.gov identifier: NCT04368065 .

Article activity feed

  1. SciScore for 10.1101/2022.01.06.22268726: (What is this?)

    Please note, not all rigor criteria are appropriate for all manuscripts.

    Table 1: Rigor

    EthicsConsent: Participants are unpaid, recruited primarily via social media and provide informed consent online.
    IRB: This study was approved by an Institutional Review Board and is registered at Clinicaltrials.gov NCT04368065 and EU PAS register EUPAS36240.
    Sex as a biological variableMultivariate odds ratios were adjusted by the following: race (white, other), gender (female, male, other [including male, transgender, not disclosed, other]), age (continuous), gender (female, male, vs. other (including not disclosed, other, transgender), education (some college or less, 4-year college degree, >4 year college degree), ethnicity of Hispanic or Latino (yes, no), access to COVID-19 vaccine proxy (early availability, high vaccination rate; early availability, low vaccination rate; late availability, high vaccination rate; late availability, low vaccination rate; medical conditions including anxiety, autoimmune disorder, blood disorder, cardiovascular disorder, depression, diabetes, hypertension, insomnia or trouble sleeping, kidney disorder, and lung disorder (yes, no).
    RandomizationWhen they reported multiple positive tests, we randomly selected one of the results as the study test result; similarly, if a participant reported multiple negative results, a random result was selected and deemed the study test result.
    Blindingnot detected.
    Power Analysisnot detected.

    Table 2: Resources

    No key resources detected.


    Results from OddPub: We did not detect open data. We also did not detect open code. Researchers are encouraged to share open data when possible (see Nature blog).


    Results from LimitationRecognizer: An explicit section about the limitations of the techniques employed in this study was not found. We encourage authors to address study limitations.

    Results from TrialIdentifier: We found the following clinical trial numbers in your paper:

    IdentifierStatusTitle
    NCT04368065RecruitingCOVID-19 Active Research Experience (CARE)


    Results from Barzooka: We did not find any issues relating to the usage of bar graphs.


    Results from JetFighter: We did not find any issues relating to colormaps.


    Results from rtransparent:
    • Thank you for including a conflict of interest statement. Authors are encouraged to include this statement when submitting to a journal.
    • Thank you for including a funding statement. Authors are encouraged to include this statement when submitting to a journal.
    • No protocol registration statement was detected.

    Results from scite Reference Check: We found no unreliable references.


    About SciScore

    SciScore is an automated tool that is designed to assist expert reviewers by finding and presenting formulaic information scattered throughout a paper in a standard, easy to digest format. SciScore checks for the presence and correctness of RRIDs (research resource identifiers), and for rigor criteria such as sex and investigator blinding. For details on the theoretical underpinning of rigor criteria and the tools shown here, including references cited, please follow this link.