Evaluation of Three Commercial and Two Non-Commercial Immunoassays for the Detection of Prior Infection to SARS-CoV-2

  1. Eric J Nilles
  2. Elizabeth W Karlson
  3. Maia Norman
  4. Tal Gilboa
  5. Stephanie Fischinger
  6. Caroline Atyeo
  7. Guohai Zhou
  8. Christopher L Bennett
  9. Nicole V Tolan
  10. Karina Oganezova
  11. David R Walt
  12. Galit Alter
  13. Daimon P Simmons
  14. Peter Schur
  15. Petr Jarolim
  16. Ann E Woolley
  17. Lindsey R Baden

Reviewed by

Read the full article See related articles

Listed in

Log in to save this article

Abstract

Background

Serological testing provides a record of prior infection with SARS-CoV-2, but assay performance requires independent assessment.

Methods

We evaluated 3 commercial (Roche Diagnostics pan-IG, and Epitope Diagnostics IgM and IgG) and 2 non-commercial (Simoa and Ragon/MGH IgG) immunoassays against 1083 unique samples that included 251 PCR-positive and 832 prepandemic samples.

Results

The Roche assay registered the highest specificity 99.6% (3/832 false positives), the Ragon/MGH assay 99.5% (4/832), the primary Simoa assay model 99.0% (8/832), and the Epitope IgG and IgM 99.0% (8/830) and 99.5% (4/830), respectively. Overall sensitivities for the Simoa, Roche pan-IG, Epitope IgG, Ragon/MGH IgG, and Epitope IgM were 92.0%, 82.9%, 82.5%, 64.5% and 47.0%, respectively. The Simoa immunoassay demonstrated the highest sensitivity among samples stratified by days postsymptom onset (PSO), <8 days PSO (57.69%) 8–14 days PSO (93.51%), 15–21 days PSO (100%), and > 21 days PSO (95.18%).

Conclusions

All assays demonstrated high to very high specificities while sensitivities were variable across assays.

Article activity feed

  1. Version published to 10.1093/jalm/jfab072
  2. Version published to 10.1101/2020.06.24.20139006v2 on medRxiv
  3. ScreenIT

    SciScore for 10.1101/2020.06.24.20139006: (What is this?)

    Please note, not all rigor criteria are appropriate for all manuscripts.

    Table 1: Rigor

    Institutional Review Board StatementIRB: Ethical considerations: The use of study samples and data was approved by the Mass General Brigham (MGB) (previously Partners Healthcare System) Institutional Review Board.
    Randomizationnot detected.
    BlindingStudy design: We conducted a head-to-head test performance study using two commercial and two non-commercial SARS-CoV-2 immunoassays where laboratories were blinded to sample group.
    Power Analysisnot detected.
    Sex as a biological variablenot detected.

    Table 2: Resources

    Antibodies
    SentencesResources
    Serological assays and protocols: We assessed the performance of four immunoassays including (i) Elecsys Anti-SARS-CoV-2 (Roche Diagnostics, Indianapolis, USA) immunoassay intended for the qualitative detection of antibodies against the nucleocapsid (NC) antigen [20] (thought to include IgG, IgM, and IgA, although IgM and IgA are not specified in product information); (ii) EDI New Coronavirus
    Anti-SARS-CoV-2
    suggested: None
    antigen [20
    suggested: None
    IgA
    suggested: None

    Results from OddPub: We did not detect open data. We also did not detect open code. Researchers are encouraged to share open data when possible (see Nature blog).

    Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:
    This study has several limitations. Although this study was based on well characterized controls, we cannot definitively extrapolate findings to other populations [28]. For example, the sensitivity of these assays was assessed in samples from RT-PCR confirmed hospitalized patients with samples collected a mean of 16 days (SD 5) after symptom onset. Given most SARS-CoV-2 infections are mild or asymptomatic and do not require hospitalization, and given that little is known about humoral kinetics >2 months post infection, the sensitivity of these assays may be lower in individuals > 2 months post-infection and/or individuals with mild or asymptomatic infections that likely mount less durable immune responses [26,28]. We did not include negative control samples from individuals with confirmed common coronaviruses, which may increase cross reactivity and false positives. Our study findings align with manufacturer data for the Roche Diagnostics’ Elecsys Anti-SARS-CoV-2 IgG immunoassay and confirm that false positives are rare, specificity is high, and sensitivity is excellent > 3 weeks after symptom onset, but lower early in the disease course and among immunocompromised individuals. The Epitope Diagnostics IgG immunoassay has lower specificity and sensitivity than data released by the manufacturer, but overall performed well with few false positives and high sensitivity. The Epitope IgM had more false positives and lower sensitivity and given IgM does not appear to rise substantia...

    Results from TrialIdentifier: No clinical trial numbers were referenced.

    Results from Barzooka: We did not find any issues relating to the usage of bar graphs.

    Results from JetFighter: We did not find any issues relating to colormaps.

    Results from rtransparent:
    • Thank you for including a conflict of interest statement. Authors are encouraged to include this statement when submitting to a journal.
    • Thank you for including a funding statement. Authors are encouraged to include this statement when submitting to a journal.
    • No protocol registration statement was detected.

    About SciScore

    SciScore is an automated tool that is designed to assist expert reviewers by finding and presenting formulaic information scattered throughout a paper in a standard, easy to digest format. SciScore checks for the presence and correctness of RRIDs (research resource identifiers), and for rigor criteria such as sex and investigator blinding. For details on the theoretical underpinning of rigor criteria and the tools shown here, including references cited, please follow this link.

  4. Version published to 10.1101/2020.06.24.20139006v1 on medRxiv