Covid-19 Vaccine Efficacy: Accuracy, Uncertainty and Projection of Cases

This article has been Reviewed by the following groups

Read the full article See related articles

Abstract

Background

Two vaccine candidates for coronavirus disease 2019 (Covid-19) have been announced by Pfizer-BioNTech and Moderna with above 90% efficacy. The efficacy of each vaccine changes between reports with no accuracy assessment.

Methods

We examined data in both vaccine trials, provided 95% confidence intervals, and projected the cases that would be prevented in communities of multi-million population.

Results

The 95% confidence intervals reveal that the true vaccine efficacy could be as low as 86% for stated efficacy of 94.4% in an interim report, indicating the inaccuracy and uncertainty of efficacy point estimate. Both vaccines achieve an efficacy above 89% by the 95% confidence interval in updated reports. The Moderna vaccine would prevent more than 50,260 cases in communities of 1 million people with 1 year exposure.

Conclusions

Point estimates of vaccine efficacy transmit limited information. Corresponding statements of uncertainty, such as confidence intervals, should be provided and included in discussions of societal impact. The Covid-19 vaccines announced to date would prevent a substantial number of cases even at lower ends of the intervals.

Article activity feed

  1. SciScore for 10.1101/2020.12.16.20248359: (What is this?)

    Please note, not all rigor criteria are appropriate for all manuscripts.

    Table 1: Rigor

    NIH rigor criteria are not applicable to paper type.

    Table 2: Resources

    No key resources detected.


    Results from OddPub: We did not detect open data. We also did not detect open code. Researchers are encouraged to share open data when possible (see Nature blog).


    Results from LimitationRecognizer: An explicit section about the limitations of the techniques employed in this study was not found. We encourage authors to address study limitations.

    Results from TrialIdentifier: No clinical trial numbers were referenced.


    Results from Barzooka: We did not find any issues relating to the usage of bar graphs.


    Results from JetFighter: We did not find any issues relating to colormaps.


    Results from rtransparent:
    • Thank you for including a conflict of interest statement. Authors are encouraged to include this statement when submitting to a journal.
    • Thank you for including a funding statement. Authors are encouraged to include this statement when submitting to a journal.
    • No protocol registration statement was detected.

    About SciScore

    SciScore is an automated tool that is designed to assist expert reviewers by finding and presenting formulaic information scattered throughout a paper in a standard, easy to digest format. SciScore checks for the presence and correctness of RRIDs (research resource identifiers), and for rigor criteria such as sex and investigator blinding. For details on the theoretical underpinning of rigor criteria and the tools shown here, including references cited, please follow this link.