Estimating the Effectiveness of First Dose of COVID-19 Vaccine Against Mortality in England: A Quasi-Experimental Study

  1. Charlotte Bermingham
  2. Jasper Morgan
  3. Daniel Ayoubkhani
  4. Myer Glickman
  5. Nazrul Islam
  6. Aziz Sheikh
  7. Jonathan Sterne
  8. A Sarah Walker
  9. Vahé Nafilyan

Reviewed by

Read the full article See related articles

Listed in

Log in to save this article

Abstract

Estimating real-world vaccine effectiveness is vital to assessing the coronavirus disease 2019 (COVID-19) vaccination program and informing the ongoing policy response. However, estimating vaccine effectiveness using observational data is inherently challenging because of the nonrandomized design and potential for unmeasured confounding. We used a regression discontinuity design to estimate vaccine effectiveness against COVID-19 mortality in England using the fact that people aged 80 years or older were prioritized for the vaccine rollout. The prioritization led to a large discrepancy in vaccination rates among people aged 80–84 years compared with those aged 75–79 at the beginning of the vaccination campaign. We found a corresponding difference in COVID-19 mortality but not in non-COVID-19 mortality, suggesting that our approach appropriately addressed the issue of unmeasured confounding factors. Our results suggest that the first vaccine dose reduced the risk of COVID-19 death by 52.6% (95% confidence limits: 15.7, 73.4) in those aged 80 years, supporting existing evidence that a first dose of a COVID-19 vaccine had a strong protective effect against COVID-19 mortality in older adults. The regression discontinuity model’s estimate of vaccine effectiveness is only slightly lower than those of previously published studies using different methods, suggesting that these estimates are unlikely to be substantially affected by unmeasured confounding factors.

Article activity feed

  1. Version published to 10.1093/aje/kwac157
  2. Version published to 10.1101/2021.07.12.21260385v2 on medRxiv
  3. ScreenIT

    SciScore for 10.1101/2021.07.12.21260385: (What is this?)

    Please note, not all rigor criteria are appropriate for all manuscripts.

    Table 1: Rigor

    EthicsIRB: Ethics approval: Ethical approval was obtained from the National Statistician’s Data Ethics Advisory Committee (NSDEC(20)12) Use of reporting guidelines: Our study was conducted following STROBE guidelines for reporting observational studies Role of funding source: The funding source played no role in the interpretation of the results
    IACUC: Ethics approval: Ethical approval was obtained from the National Statistician’s Data Ethics Advisory Committee (NSDEC(20)12) Use of reporting guidelines: Our study was conducted following STROBE guidelines for reporting observational studies Role of funding source: The funding source played no role in the interpretation of the results
    Sex as a biological variablenot detected.
    Randomizationnot detected.
    Blindingnot detected.
    Power Analysisnot detected.

    Table 2: Resources

    Antibodies
    SentencesResources
    In the survey, SARS-CoV-2 antibody levels are measured using an ELISA detecting anti-trimeric spike IgG using a fluorescence detection mechanism to 26 February 2021, then a CE-marked version of the assay, the Thermo Fisher OmniPATH 384 Combi SARS-CoV-2 IgG ELISA, using the same antigen with a colorimetric detection system.
    SARS-CoV-2
    suggested: None
    Software and Algorithms
    SentencesResources
    In the survey, SARS-CoV-2 antibody levels are measured using an ELISA detecting anti-trimeric spike IgG using a fluorescence detection mechanism to 26 February 2021, then a CE-marked version of the assay, the Thermo Fisher OmniPATH 384 Combi SARS-CoV-2 IgG ELISA, using the same antigen with a colorimetric detection system.
    Thermo Fisher OmniPATH
    suggested: None

    Results from OddPub: We did not detect open data. We also did not detect open code. Researchers are encouraged to share open data when possible (see Nature blog).

    Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:
    The main limitation of our study is that the RDD yielded an estimate of the vaccine effectiveness that is only valid for people aged around 80. It cannot be generalised to other age groups. Another limitation of the method is that people aged 75–79 years old rapidly become vaccinated, leaving a short window where the RDD can be applied. To mitigate the contamination of the control group, we estimated the discontinuity in mortality rates over a short time window (15 days), using dates where the difference in vaccination rates between people aged below and above 80 was the largest. We also estimated the discontinuity in weighted proportion of people who were vaccinated and were likely to have developed antibodies, rather than just focusing on the proportion of people who had received the vaccine. Nonetheless, as a result, our results may underestimate vaccine effectiveness. In addition, our analysis only provides an estimate of vaccine effectiveness for the first dose of COVID-19 vaccines. Another limitation is that our approach does not account for the different antibody responses in people who have previously been infected with COVID-19. Finally, the RDD approach estimates the absolute reduction in probability of COVID-19 mortality due to being vaccinated long enough to develop antibodies. To calculate vaccine effectiveness, we had to estimate the probability of COVID-19 mortality in either the untreated or treated population in order to calculate the proportion by which vacc...

    Results from TrialIdentifier: No clinical trial numbers were referenced.

    Results from Barzooka: We did not find any issues relating to the usage of bar graphs.

    Results from JetFighter: We did not find any issues relating to colormaps.

    Results from rtransparent:
    • Thank you for including a conflict of interest statement. Authors are encouraged to include this statement when submitting to a journal.
    • Thank you for including a funding statement. Authors are encouraged to include this statement when submitting to a journal.
    • No protocol registration statement was detected.

    Results from scite Reference Check: We found no unreliable references.

    About SciScore

    SciScore is an automated tool that is designed to assist expert reviewers by finding and presenting formulaic information scattered throughout a paper in a standard, easy to digest format. SciScore checks for the presence and correctness of RRIDs (research resource identifiers), and for rigor criteria such as sex and investigator blinding. For details on the theoretical underpinning of rigor criteria and the tools shown here, including references cited, please follow this link.

  4. Version published to 10.1101/2021.07.12.21260385v1 on medRxiv