Validation and clinical evaluation of a SARS-CoV-2 Surrogate Virus Neutralization Test (sVNT)

  1. Benjamin Meyer
  2. Johan Reimerink
  3. Giulia Torriani
  4. Fion Brouwer
  5. Gert-Jan Godeke
  6. Sabine Yerly
  7. Marieke Hoogerwerf
  8. Nicolas Vuilleumier
  9. Laurent Kaiser
  10. Isabella Eckerle
  11. Chantal Reusken

Reviewed by

Read the full article See related articles

Listed in

Log in to save this article

Abstract

To understand SARS-CoV-2 immunity after natural infection or vaccination, functional assays such as virus neutralizing assays are needed. So far, assays to determine SARS-CoV-2 neutralizing antibodies rely on cell-culture based infection assays either using wild type SARS-CoV-2 or pseudotyped viruses. Such assays are labour-intensive, require appropriate biosafety facilities and are difficult to standardize.

Recently, a new surrogate virus neutralisation assay (sVNT) was described that uses the principle of an ELISA to measure the neutralization capacity of anti-SARS-CoV-2 antibodies directed against the receptor binding domain.

Here, we performed an independent evaluation of the robustness, specificity and sensitivity on an extensive panel of sera from 269 PCR-confirmed COVID-19 cases and 259 unmatched samples collected before 2020 and compared it to cell-based neutralization assays. We found a high specificity of 99.2 (95%CI: 96.9-99.9) and overall sensitivity of 80.3 (95%CI: 74.9-84.8) for the sVNT. Clinical sensitivity increased between early (< 14 days post symptom onset or post diagnosis, dpos/dpd) and late sera (>14 dpos/dpd) from 75.0 (64.7-83.2) to 83.1 (76.5-88.1). Also, higher severity was associated with an increase in clinical sensitivity. Upon comparison with cell-based neutralisation assays we determined an analytical sensitivity of 74.3 (56.4-86.9) and 98.2 (89.4-99.9) for titres ≥10 to < 40 and ≥40 to < 160, respectively. Only samples with a titre ≥160 were always positive in the sVNT.

In conclusion, the sVNT can be used as an additional assay to determine the immune status of COVID-19 infected of vaccinated individuals but its value needs to be assessed for the specific context of use.

Article activity feed

  1. ScreenIT

    SciScore for 10.1101/2020.09.21.20191288: (What is this?)

    Please note, not all rigor criteria are appropriate for all manuscripts.

    Table 1: Rigor

    Institutional Review Board StatementIRB: Sample collection: RIVM: sera from common CoV cases and non-CoV respiratory cases were partially obtained from a previous study approved by the ethics committee of the National Institute of Public Health and the Environment (METC Noord-Holland, http://www.trialregister.nl; NTR3386 and 481810 and partially from anonymized leftover serum from routine diagnostics for respiratory pathogens or SARS-CoV-2 for which ethical approval was waived by the ethics committees of Brabant and Utrecht (NW2020-31 and NL13529.041.06; 06/282).
    Randomizationnot detected.
    Blindingnot detected.
    Power Analysisnot detected.
    Sex as a biological variablenot detected.
    Cell Line AuthenticationContamination: Influenza A virus (n=3), human metapneumovirus (HMPV) (n=4), respiratory syncytial virus (RSV) B (n=1), RSV A + HMPV (n=1), hemophilus influenza (n =1), mycoplasma pneumonia (n=1) and rhinovirus (n=3).

    Table 2: Resources

    Experimental Models: Cell Lines
    SentencesResources
    Vero E6 were then infected with 100µl of virus-serum mixtures.
    Vero E6
    suggested: None
    Vero-E6 cells were added in a concentration of 2 × 104 cells per well and incubated for three days at 35°C in an incubator with 5% CO2.
    Vero-E6
    suggested: None
    Software and Algorithms
    SentencesResources
    GFP positive infected cells were counted with ImageXpress® Micro Widefield High Content Screening System (Molecular Devices) and data analyzed with MetaXpress 5.1.0.41 software.
    MetaXpress
    suggested: (MetaXpress, RRID:SCR_016654)
    Statistical analysis: Statistical analysis was performed with GraphPad Prism version 8.4.3 using Mann-Whitney or Kruskal-Wallis test with Dunn’s multiple comparison test where appropriate.
    GraphPad Prism
    suggested: (GraphPad Prism, RRID:SCR_002798)
    Calculation of sensitivity, specificity and 95% confidence intervals (95%CI) was done using the VassarStats statistical toolbox (http://www.vassarstats.net/).
    VassarStats
    suggested: (VassarStats, RRID:SCR_010263)

    Results from OddPub: We did not detect open data. We also did not detect open code. Researchers are encouraged to share open data when possible (see Nature blog).

    Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:
    A limitation of the sVNT is the restriction to RBD binding antibodies. It has been shown that SARS-CoV-2 infection does not only induce antibodies against the RBD or the S1 domain, but also against the S2 domain as well as against N19. While antibodies directed against the N protein are most likely non-neutralizing, antibodies directed against the N-terminal domain of S1 (outside of the RBD) have shown neutralizing potential20. In addition, for SARS-CoV-1 also antibodies directed at the S2 domain show neutralizing capabilities21. Although S2 domain-mediated neutralisation remains to be confirmed for SARS-CoV-2, assessing the neutralizing ability with the sVNT might indeed miss the presence of virus neutralizing capabilities directed outside of the RBD. Furthermore, all of the sera that were reactive in the sVNT but not in the VNT50/PNT50 were reactive in the Wantai total Ig ELISA that targets RBD as well (data not shown). This indicates that the activity measured in the sVNT is likely due to antibodies directed against the RBD but without a virus neutralising capacity that leads to at least 50% reduction of infected cells. Another explanation for such false positivity with respect to neutralizing capability might be the presence of anti-ACE2 autoantibodies as described for specific patient groups22,23. The observation of false positives and the observed lower sensitivity (74%) of the sVNT in comparison to conventional tests in samples with neutralizing antibody titres ≥10 and...

    Results from TrialIdentifier: No clinical trial numbers were referenced.

    Results from Barzooka: We did not find any issues relating to the usage of bar graphs.

    Results from JetFighter: We did not find any issues relating to colormaps.

    Results from rtransparent:
    • Thank you for including a conflict of interest statement. Authors are encouraged to include this statement when submitting to a journal.
    • Thank you for including a funding statement. Authors are encouraged to include this statement when submitting to a journal.
    • No protocol registration statement was detected.

    About SciScore

    SciScore is an automated tool that is designed to assist expert reviewers by finding and presenting formulaic information scattered throughout a paper in a standard, easy to digest format. SciScore checks for the presence and correctness of RRIDs (research resource identifiers), and for rigor criteria such as sex and investigator blinding. For details on the theoretical underpinning of rigor criteria and the tools shown here, including references cited, please follow this link.

  2. Version published to 10.1101/2020.09.21.20191288v1 on medRxiv
  3. Version published to 10.1080/22221751.2020.1835448