Analytic comparison between three high-throughput commercial SARS-CoV-2 antibody assays reveals minor discrepancies in a high-incidence population

  1. Gheyath K. Nasrallah
  2. Soha R. Dargham
  3. Farah Shurrab
  4. Duaa W. Al-Sadeq
  5. Hadeel Al-Jighefee
  6. Hiam Chemaitelly
  7. Zaina Al Kanaani
  8. Abdullatif Al Khal
  9. Einas Al Kuwari
  10. Peter Coyle
  11. Andrew Jeremijenko
  12. Anvar Hassan Kaleeckal
  13. Ali Nizar Latif
  14. Riyazuddin Mohammad Shaik
  15. Hanan F. Abdul Rahim
  16. Hadi M. Yassine
  17. Mohamed G. Al Kuwari
  18. Hamda Qotba
  19. Hamad Eid Al Romaihi
  20. Patrick Tang
  21. Roberto Bertollini
  22. Mohamed H. Al-Thani
  23. Asmaa A. Althani
  24. Laith J. Abu-Raddad

Reviewed by

Read the full article See related articles

Listed in

Log in to save this article

Abstract

Performance of three automated commercial serological IgG-based assays was investigated for assessing SARS-CoV-2 “ever” (past or current) infection in a population-based sample in a high exposure setting. PCR and serological testing was performed on 394 individuals. SARS-CoV-2-IgG seroprevalence was 42.9% (95% CI 38.1–47.8%), 40.6% (95% CI 35.9–45.5%), and 42.4% (95% CI 37.6–47.3%) using the CL-900i, VidasIII, and Elecsys assays, respectively. Between the three assays, overall, positive, and negative percent agreements ranged between 93.2–95.7%, 89.3–92.8%, and 93.8–97.8%, respectively; Cohen’s kappa statistic ranged from 0.86 to 0.91; and 35 specimens (8.9%) showed discordant results. Among all individuals, 12.5% (95% CI 9.6–16.1%) had current infection, as assessed by PCR. Of these, only 34.7% (95% CI 22.9–48.7%) were seropositive by at least one assay. A total of 216 individuals (54.8%; 95% CI 49.9–59.7%) had evidence of ever infection using antibody testing and/or PCR during or prior to this study. Of these, only 78.2%, 74.1%, and 77.3% were seropositive in the CL-900i, VidasIII, and Elecsys assays, respectively. All three assays had comparable performance and excellent agreement, but missed at least 20% of individuals with past or current infection. Commercial antibody assays can substantially underestimate ever infection, more so when infection rates are high.

Article activity feed

  1. Version published to 10.1038/s41598-021-91235-x
  2. ScreenIT

    SciScore for 10.1101/2020.12.14.20248163: (What is this?)

    Please note, not all rigor criteria are appropriate for all manuscripts.

    Table 1: Rigor

    Institutional Review Board StatementIRB: The research work was approved by the ethics review boards at HMC, QU, and Weill Cornell Medicine-Qatar.
    Randomizationnot detected.
    Blindingnot detected.
    Power Analysisnot detected.
    Sex as a biological variablenot detected.

    Table 2: Resources

    Antibodies
    SentencesResources
    seroprevalence (IgG antibodies) and current-infection prevalence (using polymerase chain reaction [PCR] testing) in the wider population of craft and manual workers who constitute 60% of the population of Qatar [11].
    IgG
    suggested: None
    The Roche Elecsys® Anti SARS-CoV-2 (“Elecsys” in short form) assay, our reference assasy, uses a recombinant protein representing the nucleocapsid (N) antigen for the determination of IgG antibodies against SARS-CoV-2 [5].
    Anti SARS-CoV-2
    suggested: None
    SARS-CoV-2 [ 5
    suggested: None
    The Mindray CL-900i® anti-SARS-CoV-2 IgG (“CL-900i” in short form) assay uses paramagnetic microplates coated with recombinant nucleocapsid (N) and spike (S) antigens for the determination of anti-SARS-CoV-2 IgG antibodies [6].
    anti-SARS-CoV-2 IgG
    suggested: None
    Software and Algorithms
    SentencesResources
    Calculations were conducted using Microsoft Excel.
    Microsoft Excel
    suggested: (Microsoft Excel, RRID:SCR_016137)

    Results from OddPub: We did not detect open data. We also did not detect open code. Researchers are encouraged to share open data when possible (see Nature blog).

    Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:
    This study has some limitations. Two out of the 394 participants included in the study declined PCR testing (but not serological testing) at time of specimen collection. The performance of these antibody assays was compared to each other (and to PCR testing), but not to a gold standard test of seropositivity, as such a test was not available to study investigators. Therefore, we were unable to measure ever infection prevalence to a gold standard, and use this to compare the performance of each assay to the gold standard, nor to assess the sensitivity and specificity of each assay in the study sample. The specificity of the Elecsys assay has previously been reported to be 99.98% and the sensitivity to be 98.80% on day 14 after PCR diagnosis [5]. A validation study by Public Health England reported a specificity of 100% and a sensitivity of 83.9% for the same assay [22]. As for the remaining assays, specificity and sensitivity were reported at 94.9% and 82.2%, respectively, for the CL-900i assay [23], and at 99.9% and 88.6%, respectively, for the VidasIII assay [7]. In conclusion, all three assays had comparable performance and excellent agreement when used in a high SARS-CoV-2 exposure setting, but still missed at least 20% of cases with laboratory-confirmed evidence of ever infection. This suggests that current growing use of commercial antibody assays to assess ever infection in population-based surveys, especially at times of high SARS-CoV-2 incidence when many infections a...

    Results from TrialIdentifier: No clinical trial numbers were referenced.

    Results from Barzooka: We did not find any issues relating to the usage of bar graphs.

    Results from JetFighter: We did not find any issues relating to colormaps.

    Results from rtransparent:
    • Thank you for including a conflict of interest statement. Authors are encouraged to include this statement when submitting to a journal.
    • Thank you for including a funding statement. Authors are encouraged to include this statement when submitting to a journal.
    • Thank you for including a protocol registration statement.

    About SciScore

    SciScore is an automated tool that is designed to assist expert reviewers by finding and presenting formulaic information scattered throughout a paper in a standard, easy to digest format. SciScore checks for the presence and correctness of RRIDs (research resource identifiers), and for rigor criteria such as sex and investigator blinding. For details on the theoretical underpinning of rigor criteria and the tools shown here, including references cited, please follow this link.

  3. Version published to 10.1101/2020.12.14.20248163v1 on medRxiv