Evaluation of High-Throughput SARS-CoV-2 Serological Assays in a Longitudinal Cohort of Patients with Mild COVID-19: Clinical Sensitivity, Specificity, and Association with Virus Neutralization Test

  1. Antonin Bal
  2. Bruno Pozzetto
  3. Mary-Anne Trabaud
  4. Vanessa Escuret
  5. Muriel Rabilloud
  6. Carole Langlois-Jacques
  7. Adèle Paul
  8. Nicolas Guibert
  9. Constance D’Aubarède-Frieh
  10. Amélie Massardier-Pilonchery
  11. Nicole Fabien
  12. David Goncalves
  13. André Boibieux
  14. Florence Morfin-Sherpa
  15. Virginie Pitiot
  16. François Gueyffier
  17. Bruno Lina
  18. Jean-Baptiste Fassier
  19. Sophie Trouillet-Assant
  20. COVID SER Study Group

Reviewed by

Read the full article See related articles

Discuss this preprint

Start a discussion What are Sciety discussions?

Listed in

Log in to save this article

Abstract

Background

The association between SARS-CoV-2 commercial serological assays and virus neutralization test (VNT) has been poorly explored in mild patients with COVID-19.

Methods

439 serum specimens were longitudinally collected from 76 healthcare workers with RT-PCR-confirmed COVID-19. The clinical sensitivity (determined weekly) of 9 commercial serological assays were evaluated. Clinical specificity was assessed using 69 pre-pandemic sera. Correlation, agreement, and concordance with the VNT were also assessed on a subset of 170 samples. Area under the ROC curve (AUC) was estimated at 2 neutralizing antibody titers.

Results

The Wantai Total Ab assay targeting the receptor binding domain (RBD) within the S protein presented the best sensitivity at different times during the course of disease. The clinical specificity was greater than 95% for all tests except for the Euroimmun IgA assay. The overall agreement with the presence of neutralizing antibodies ranged from 62.2% (95%CI; 56.0–68.1) for bioMérieux IgM to 91.2% (87.0–94.2) for Siemens. The lowest negative percent agreement (NPA) was found with the Wantai Total Ab assay (NPA 33% (21.1–48.3)). The NPA for other total Ab or IgG assays targeting the S or the RBD was 80.7% (66.7–89.7), 90.3% (78.1–96.1), and 96.8% (86.8–99.3) for Siemens, bioMérieux IgG, and DiaSorin, respectively. None of the commercial assays have sufficient performance to detect a neutralizing titer of 80 (AUC < 0.76).

Conclusions

Although some assays show a better agreement with VNT than others, the present findings emphasize that commercialized serological tests, including those targeting the RBD, cannot substitute a VNT for the assessment of functional antibody response.

Article activity feed

  1. Version published to 10.1093/clinchem/hvaa336
  2. ScreenIT

    SciScore for 10.1101/2020.09.30.20194290: (What is this?)

    Please note, not all rigor criteria are appropriate for all manuscripts.

    Table 1: Rigor

    Institutional Review Board StatementConsent: Written informed consent was obtained from all participants; ethics approval was obtained from the national review board for biomedical research in April 2020 (Comité de Protection des Personnes Sud Méditerranée I, Marseille, France; ID RCB 2020-A00932-37), and the study was registered on ClinicalTrials.gov (NCT04341142).
    IRB: Written informed consent was obtained from all participants; ethics approval was obtained from the national review board for biomedical research in April 2020 (Comité de Protection des Personnes Sud Méditerranée I, Marseille, France; ID RCB 2020-A00932-37), and the study was registered on ClinicalTrials.gov (NCT04341142).
    Randomizationnot detected.
    Blindingnot detected.
    Power Analysisnot detected.
    Sex as a biological variablenot detected.
    Cell Line Authenticationnot detected.

    Table 2: Resources

    Experimental Models: Cell Lines
    SentencesResources
    After gentle shaking and a contact of 30 minutes at room temperature in plastic microplates, 150 µL of the mix was transferred into 96-well microplates covered with Vero E6 cells.
    Vero E6
    suggested: RRID:CVCL_XD71)
    Software and Algorithms
    SentencesResources
    A total of 9 serological assays (Abbott, DiaSorin, Siemens, Bio-Rad, Wantai Total and IgM, bioMérieux IgG and IgM, Euroimmun IgA) were investigated according to the protocol recommended by each manufacturer (characteristics are summarized in Table 1).
    Abbott
    suggested: (Abbott, RRID:SCR_010477)
    Statistical analyses were carried out using SAS software, version 9.4 (Copyright (c) 2002-2003 by SAS Institute Inc.
    SAS
    suggested: (SASqPCR, RRID:SCR_003056)
    SAS Institute
    suggested: (Statistical Analysis System, RRID:SCR_008567)

    Results from OddPub: We did not detect open data. We also did not detect open code. Researchers are encouraged to share open data when possible (see Nature blog).

    Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:
    The present study does, however, have certain limitations. For instance, specificity was not been extensively studied; yet the Euroimmun IgA assay seemed to have the worst specificity, which is consistent with previous studies reporting a lack of specificity for this assay (5,6,14). In addition, the performance of other notable commercial assays such as Euroimmun IgG or Roche Ig Total were not assessed. Second, not all the samples were systematically tested by VNT, in-line with the labor-intensive nature of this method. Finally, the size of the tested population remains small contributing to wide CI which limits the interpretation and extrapolation of the results. The results presented herein confirm that, the Wantai Total Ab assay presented the higher sensitivity to detect SARS-CoV-2 antibodies after exposure. For the screening of neutralizing antibodies in serum specimens, optimized cut-off maximizing the NPA need to be established as previously suggest for the Wantai Total Ab assay (14). However, the data presented herein suggest that other tests targeting the S protein as Siemens, DiaSorin or bioMérieux IgG might be more useful for this indication. These tests or others cannot substitute a VNT for assessing functional antibody response; neutralizing assays remain the gold standard and easy-to-use tests, such as those based on pseudoviruses (6,22,35), should be developed and standardized. Furthermore, the recent development of surrogate virus neutralization tests based on ...

    Results from TrialIdentifier: We found the following clinical trial numbers in your paper:

    IdentifierStatusTitle
    NCT04341142RecruitingAssessment of Serological Techniques for Screening Patients …

    Results from Barzooka: We did not find any issues relating to the usage of bar graphs.

    Results from JetFighter: We did not find any issues relating to colormaps.

    Results from rtransparent:
    • Thank you for including a conflict of interest statement. Authors are encouraged to include this statement when submitting to a journal.
    • Thank you for including a funding statement. Authors are encouraged to include this statement when submitting to a journal.
    • No protocol registration statement was detected.

    About SciScore

    SciScore is an automated tool that is designed to assist expert reviewers by finding and presenting formulaic information scattered throughout a paper in a standard, easy to digest format. SciScore checks for the presence and correctness of RRIDs (research resource identifiers), and for rigor criteria such as sex and investigator blinding. For details on the theoretical underpinning of rigor criteria and the tools shown here, including references cited, please follow this link.

  3. Version published to 10.1101/2020.09.30.20194290 on medRxiv