Calibration of two validated SARS-CoV-2 pseudovirus neutralization assays for COVID-19 vaccine evaluation

  1. Yunda Huang
  2. Oleg Borisov
  3. Jia Jin Kee
  4. Lindsay N. Carpp
  5. Terri Wrin
  6. Suqin Cai
  7. Marcella Sarzotti-Kelsoe
  8. Charlene McDanal
  9. Amanda Eaton
  10. Rolando Pajon
  11. John Hural
  12. Christine M. Posavad
  13. Katherine Gill
  14. Shelly Karuna
  15. Lawrence Corey
  16. M. Juliana McElrath
  17. Peter B. Gilbert
  18. Christos J. Petropoulos
  19. David C. Montefiori

Reviewed by

Read the full article See related articles

Listed in

Log in to save this article

Abstract

Vaccine-induced neutralizing antibodies (nAbs) are key biomarkers considered to be associated with vaccine efficacy. In United States government-sponsored phase 3 efficacy trials of COVID-19 vaccines, nAbs are measured by two different validated pseudovirus-based SARS-CoV-2 neutralization assays, with each trial using one of the two assays. Here we describe and compare the nAb titers obtained in the two assays. We observe that one assay consistently yielded higher nAb titers than the other when both assays were performed on the World Health Organization’s anti-SARS-CoV-2 immunoglobulin International Standard, COVID-19 convalescent sera, and mRNA-1273 vaccinee sera. To overcome the challenge this difference in readout poses in comparing/combining data from the two assays, we evaluate three calibration approaches and show that readouts from the two assays can be calibrated to a common scale. These results may aid decision-making based on data from these assays for the evaluation and licensure of new or adapted COVID-19 vaccines.

Article activity feed

  1. Version published to 10.1038/s41598-021-03154-6
  2. ScreenIT

    SciScore for 10.1101/2021.09.09.21263049: (What is this?)

    Please note, not all rigor criteria are appropriate for all manuscripts.

    Table 1: Rigor

    Ethicsnot detected.
    Sex as a biological variablenot detected.
    Randomizationnot detected.
    BlindingThe laboratory was blinded to the clinical status of the donors but not to HIV-1 infection status.
    Power Analysisnot detected.
    Cell Line Authenticationnot detected.

    Table 2: Resources

    Antibodies
    SentencesResources
    WHO anti SARS-CoV-2 immunoglobulin International Standard (IS) sample: In December 2020, the WHO released a well-characterized international standard for the purpose of improving comparability of results among different assays in different laboratories and reducing interlaboratory variability of anti-SARS-CoV-2 antibody assays.
    anti SARS-CoV-2
    suggested: None
    anti-SARS-CoV-2
    suggested: None
    The intended use of the International Standard is for the calibration and harmonization of serological assays detecting anti-SARS-CoV-2 neutralizing antibodies.
    anti-SARS-CoV-2 neutralizing antibodies.
    suggested: (Creative Diagnostics Cat# CABT-CS064, RRID:AB_2891088)
    Experimental Models: Cell Lines
    SentencesResources
    Transfection mixtures were added to pre-seeded HEK 293T/17 cells in T-75 flasks containing 12 ml of growth medium and incubated for 16-20 h at 37°C.
    HEK 293T/17
    suggested: ATCC Cat# CRL-11268, RRID:CVCL_1926)
    293T/ACE2-MF cells were detached from T75 culture flasks using TrypLE Select Enzyme solution, suspended in growth medium (100,000 cells/ml) and immediately added to all wells (10,000 cells in 100 µL of growth medium per well).
    293T/ACE2-MF
    suggested: None
    Pseudovirus stock was produced in HEK 293 cells via a calcium phosphate transfection using a combination of spike plasmid (pCXAS-SARS-CoV-2-D614G) and lentiviral backbone plasmid (F-lucP.CNDOΔU3)
    HEK 293
    suggested: None
    Recombinant DNA
    SentencesResources
    Pseudovirions were produced in HEK293T/17 cells (ATCC cat. no. CRL-11268) by transfection using Fugene 6 (Promega Cat#E2692) and a combination of spike plasmid, lentiviral backbone plasmid (pCMV ΔR8.2) and firefly Luc reporter gene plasmid (pHR’ CMV Luc)36 in a 1:17:17 ratio in Opti-MEM (Life Technologies).
    pCMV ΔR8.2
    suggested: None
    A codon-optimized version of the full-length spike gene of the Wuhan-1 SARS-CoV-2 strain (MN908947.3) (GenScript) was cloned into the Monogram proprietary env expression vector, pCXAS-PXMX, for use in the assay.
    pCXAS-PXMX
    suggested: None
    Pseudovirus stock was produced in HEK 293 cells via a calcium phosphate transfection using a combination of spike plasmid (pCXAS-SARS-CoV-2-D614G) and lentiviral backbone plasmid (F-lucP.CNDOΔU3)
    pCXAS-SARS-CoV-2-D614G
    suggested: None
    Software and Algorithms
    SentencesResources
    All mutations were confirmed by full-length spike gene sequencing by Sanger Sequencing, using Sequencher and SnapGene for sequence analyses.
    Sequencher
    suggested: (Sequencher, RRID:SCR_001528)
    SnapGene
    suggested: (SnapGene, RRID:SCR_015052)

    Results from OddPub: We did not detect open data. We also did not detect open code. Researchers are encouraged to share open data when possible (see Nature blog).

    Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:
    There are several limitations of this study. First, for evaluating the performance of the calibration approaches, we only considered a limited number of samples from trials of the mRNA-1273 vaccine. This potentially limits our ability to generalize findings from the calibration study to other COVID-19 vaccine platforms. We plan to assay a subset of samples from phase 3 trials of other COVID-19 vaccines to further validate the calibration approaches when those data become available. Second, to develop the calibration algorithms, both labs must have their assays performed on a common pool of independent samples that are expected to cover a similar dynamic range as the readouts requiring calibration. Access to such a common pool may be challenging for many labs. In that case, Approach 1 might be the most feasible option as long as the WHO IS sample is available in sufficient quantity. Third, for all three approaches, we focused the calibration only for titers above the lower limit of detection (LLoD) of the reference assays. Methods that account for calibration of responses below the LLoD, especially in the context of two assays having different LLoDs, are currently under development. Fourth, this study only concerns data from two specific labs measuring nAbs. Further validation is needed to investigate the performance of the three approaches applied to the calibration of other assays and/or immune responses. Although the described calibration approaches are only applied to the ...

    Results from TrialIdentifier: We found the following clinical trial numbers in your paper:

    IdentifierStatusTitle
    NCT04283461Active, not recruitingSafety and Immunogenicity Study of 2019-nCoV Vaccine (mRNA-1…
    NCT04470427Active, not recruitingA Study to Evaluate Efficacy, Safety, and Immunogenicity of …
    NCT04403880Active, not recruitingCharacterizing SARS-CoV-2-specific Immunity in Individuals W…

    Results from Barzooka: We did not find any issues relating to the usage of bar graphs.

    Results from JetFighter: We did not find any issues relating to colormaps.

    Results from rtransparent:
    • Thank you for including a conflict of interest statement. Authors are encouraged to include this statement when submitting to a journal.
    • Thank you for including a funding statement. Authors are encouraged to include this statement when submitting to a journal.
    • No protocol registration statement was detected.

    Results from scite Reference Check: We found no unreliable references.

    About SciScore

    SciScore is an automated tool that is designed to assist expert reviewers by finding and presenting formulaic information scattered throughout a paper in a standard, easy to digest format. SciScore checks for the presence and correctness of RRIDs (research resource identifiers), and for rigor criteria such as sex and investigator blinding. For details on the theoretical underpinning of rigor criteria and the tools shown here, including references cited, please follow this link.

  3. Version published to 10.1101/2021.09.09.21263049v1 on medRxiv
  4. Version published to 10.21203/rs.3.rs-862572/v1 on Research Square