Rapid evaluation of COVID-19 vaccine effectiveness against symptomatic infection with SARS-CoV-2 variants by analysis of genetic distance

  1. Lirong Cao
  2. Jingzhi Lou
  3. See Yeung Chan
  4. Hong Zheng
  5. Caiqi Liu
  6. Shi Zhao
  7. Qi Li
  8. Chris Ka Pun Mok
  9. Renee Wan Yi Chan
  10. Marc Ka Chun Chong
  11. William Ka Kei Wu
  12. Zigui Chen
  13. Eliza Lai Yi Wong
  14. Paul Kay Sheung Chan
  15. Benny Chung Ying Zee
  16. Eng Kiong Yeoh
  17. Maggie Haitian Wang

Reviewed by

Read the full article See related articles

Discuss this preprint

Start a discussion What are Sciety discussions?

Listed in

Log in to save this article

Abstract

Timely evaluation of the protective effects of Coronavirus Disease 2019 (COVID-19) vaccines against severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) variants of concern is urgently needed to inform pandemic control planning. Based on 78 vaccine efficacy or effectiveness (VE) data from 49 studies and 1,984,241 SARS-CoV-2 sequences collected from 31 regions, we analyzed the relationship between genetic distance (GD) of circulating viruses against the vaccine strain and VE against symptomatic infection. We found that the GD of the receptor-binding domain of the SARS-CoV-2 spike protein is highly predictive of vaccine protection and accounted for 86.3% ( P  = 0.038) of the VE change in a vaccine platform-based mixed-effects model and 87.9% ( P  = 0.006) in a manufacturer-based model. We applied the VE-GD model to predict protection mediated by existing vaccines against new genetic variants and validated the results by published real-world and clinical trial data, finding high concordance of predicted VE with observed VE. We estimated the VE against the Delta variant to be 82.8% (95% prediction interval: 68.7–96.0) using the mRNA vaccine platform, closely matching the reported VE of 83.0% from an observational study. Among the four sublineages of Omicron, the predicted VE varied between 11.9% and 33.3%, with the highest VE predicted against BA.1 and the lowest against BA.2, using the mRNA vaccine platform. The VE-GD framework enables predictions of vaccine protection in real time and offers a rapid evaluation method against novel variants that may inform vaccine deployment and public health responses.

Article activity feed

  1. Version published to 10.1038/s41591-022-01877-1
  2. ScreenIT

    SciScore for 10.1101/2021.04.22.21254079: (What is this?)

    Please note, not all rigor criteria are appropriate for all manuscripts.

    Table 1: Rigor

    NIH rigor criteria are not applicable to paper type.

    Table 2: Resources

    Software and Algorithms
    SentencesResources
    Multiple sequence alignment was performed using MAFFT (version 7).
    MAFFT
    suggested: (MAFFT, RRID:SCR_011811)

    Results from OddPub: We did not detect open data. We also did not detect open code. Researchers are encouraged to share open data when possible (see Nature blog).

    Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:
    This study has several limitations. Although the current model reached good statistical significance, the complexity of the model is restricted by the sample size of the available studies reporting VE. Thus, the platform effect and population effect on vaccine performance cannot be addressed. Secondly, the existing genetic mismatch was evaluated by measuring genetic distance, and other chemical and spatial structure changes associated with amino acid substitutions were not considered. Nevertheless, we demonstrated that genetic mismatch explains a sizable portion of VE deviations in clinical trials. These results imply that it is essential to monitor newly arising variants, and COVID-19 vaccines will need to be updated periodically to avoid potential loss of efficacy.

    Results from TrialIdentifier: No clinical trial numbers were referenced.

    Results from Barzooka: We did not find any issues relating to the usage of bar graphs.

    Results from JetFighter: We did not find any issues relating to colormaps.

    Results from rtransparent:
    • Thank you for including a conflict of interest statement. Authors are encouraged to include this statement when submitting to a journal.
    • Thank you for including a funding statement. Authors are encouraged to include this statement when submitting to a journal.
    • Thank you for including a protocol registration statement.

    About SciScore

    SciScore is an automated tool that is designed to assist expert reviewers by finding and presenting formulaic information scattered throughout a paper in a standard, easy to digest format. SciScore checks for the presence and correctness of RRIDs (research resource identifiers), and for rigor criteria such as sex and investigator blinding. For details on the theoretical underpinning of rigor criteria and the tools shown here, including references cited, please follow this link.

  3. Version published to 10.1101/2021.04.22.21254079 on medRxiv