Performance of Abbott Architect, Ortho Vitros, and Euroimmun Assays in Detecting Prior SARS-CoV-2 Infection

  1. Shiwani Mahajan
  2. Carrie A. Redlich
  3. Adam V. Wisnewski
  4. Louis E. Fazen
  5. Lokinendi V. Rao
  6. Karthik Kuppusamy
  7. Albert I. Ko
  8. Harlan M. Krumholz

Reviewed by

Read the full article See related articles

Discuss this preprint

Start a discussion What are Sciety discussions?

Listed in

Log in to save this article

Abstract

Background

Several serological assays have been developed to detect anti-SARS-CoV-2 IgG antibodies, but evidence about their comparative performance is limited. We sought to assess the sensitivity of four anti-SARS-CoV-2 IgG enzyme-linked immunosorbent assays (ELISA) in individuals with evidence of prior SARS-CoV-2 infection.

Methods

We obtained sera from 36 individuals with PCR-confirmed SARS-CoV-2 infection between March and May 2020. We evaluated samples collected at around 21 days (±14 days) after their initial PCR test using 3 commercially available ELISA assays, two anti-spike (Ortho- Clinical Diagnostics Vitros, and Euroimmun) and one anti-nucleocapsid (Abbott Architect), and a Yale-developed anti-spike ELISA test. We determined the sensitivity of the tests and compared their results. The Euroimmun and Yale ELISA had an equivocal and indeterminate category, which were considered as both negative and positive.

Results

Among the 36 individuals with SARS-CoV-2 infection, mean age was 43 (±13) years and 19 (53%) were female. The sensitivities of the tests were not significantly different (Abbott Architect, Ortho Vitros, Euroimmmun, and Yale assays: 86% (95% confidence interval [CI], 71– 95), 94% (95% CI, 81–99), 86% (95% CI, 71–95), and 94% (95% CI, 81–99), respectively; p- value=0.464). The sensitivities of the Euroimmun and Yale ELISA tests increased when the equivocal/indeterminate results were considered positive (97% [95% CI, 85–100] and 100% [95% CI, 90–100], respectively), but were not significantly different from other tests (p=0.082). The cross-correlation coefficient ranged from 0.85–0.98 between three anti-spike protein assays (Ortho Vitros, Euroimmun, Yale) and was 0.58–0.71 between the three anti-spike protein assays and the anti-nucleocapsid assay (Abbott).

Conclusion

The sensitivities of four anti-SARS-CoV-2 protein assays did not significantly differ, although the sample size was small. Sensitivity also depended on the interpretation of equivocal and indeterminate results. The strongest correlations were present for the three anti- spike proteins assays. These findings suggest that individual test characteristics and the correlation between different tests should be considered when comparing or aggregating data across different populations studies for serologic surveillance of past SARS-CoV-2 infection.

Article activity feed

  1. ScreenIT

    SciScore for 10.1101/2020.07.29.20164343: (What is this?)

    Please note, not all rigor criteria are appropriate for all manuscripts.

    Table 1: Rigor

    Institutional Review Board StatementConsent: All individuals provided informed consent and samples were collected in accordance with protocols approved by the Institutional Review Board at Yale University.
    IRB: All individuals provided informed consent and samples were collected in accordance with protocols approved by the Institutional Review Board at Yale University.
    Randomizationnot detected.
    BlindingSamples had previously been assayed on a laboratory developed test at Yale, an ELISA against the spike antigen, but samples were selected blinded from these results.
    Power Analysisnot detected.
    Sex as a biological variablenot detected.

    Table 2: Resources

    Software and Algorithms
    SentencesResources
    Approximately 450uL of serum from each of the 36 samples was sent from Yale to Quest Diagnostics for testing on their 3 commercially-available SARS-CoV-2 IgG assays, namely – Abbott Architect SARS-CoV-2 IgG test, Ortho-Clinical Diagnostics Vitros anti- SARS-CoV-2 IgG test, and Euroimmun SARS-CoV-2 ELISA (IgG).
    Abbott Architect
    suggested: (Abbott ARCHITECT i1000sr System, RRID:SCR_019328)
    The following cutoffs were used for reporting of results according to the manufacturer’s instructions: 1) Abbott Architect IgG assay3: <1.4 = negative and ≥1.4 = positive; 2) Ortho Vitros IgG assay4: <1.00 = negative and ≥1.00 = positive; and 3) Euroimmun IgG assay5: <0.8 = negative, ≥0.8 to <1.1 = equivocal, and ≥1.1 = positive.
    Abbott Architect IgG
    suggested: (Resource Identification Portal, RRID:SCR_004098)

    Results from OddPub: We did not detect open data. We also did not detect open code. Researchers are encouraged to share open data when possible (see Nature blog).

    Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:
    Our study has several limitations. Our comparative study focused on the distribution and correlation of anti-SARS-CoV-2 IgG ELISA results among individuals who had confirmed COVID-19, and we were not able to evaluate the specificities of the tests. In addition, our sample size limited our power to detect clinically meaningful differences in sensitivities. Another limitation is that we had limited information on the clinical characteristics of these individuals, which could have affected the nature of antibody response. Nevertheless, this study provides important information on the correlations between 3 commercially available serology tests, as well as the Yale anti-spike ELISA, and also information on the sensitivity of these tests, which can help guide future research on the seroprevalence of IgG antibodies to SARS-CoV-2. In conclusion, this study shows that the 3 commercial SARS-CoV-2 IgG assays (Abbott Architect, Ortho Vitros, and Euroimmun) and the Yale anti-spike ELISA, had sensitivities that were not significantly different. Additionally, the 3 assays based on the spike protein showed the strongest correlations in detecting SARS-CoV-2 antibodies and had a weaker correlation with the serology test based on the nucleocapsid protein. As such, serologic surveillance of past SARS-CoV-2 infection should consider the individual test characteristics and the correlation between different tests when comparing or aggregating data across different populations.

    Results from TrialIdentifier: No clinical trial numbers were referenced.

    Results from Barzooka: We did not find any issues relating to the usage of bar graphs.

    Results from JetFighter: We did not find any issues relating to colormaps.

    Results from rtransparent:
    • Thank you for including a conflict of interest statement. Authors are encouraged to include this statement when submitting to a journal.
    • Thank you for including a funding statement. Authors are encouraged to include this statement when submitting to a journal.
    • No protocol registration statement was detected.

    About SciScore

    SciScore is an automated tool that is designed to assist expert reviewers by finding and presenting formulaic information scattered throughout a paper in a standard, easy to digest format. SciScore checks for the presence and correctness of RRIDs (research resource identifiers), and for rigor criteria such as sex and investigator blinding. For details on the theoretical underpinning of rigor criteria and the tools shown here, including references cited, please follow this link.

  2. ScreenIT

    SciScore for 10.1101/2020.07.29.20164343: (What is this?)

    Please note, not all rigor criteria are appropriate for all manuscripts.

    Table 1: Rigor

    Institutional Review Board StatementAll individuals provided informed consent and samples were collected in accordance with protoco approved by the Institutional Review Board at Yale University.Randomizationnot detected.BlindingSamples had previously been assayed on a laboratory developed test at Yale an ELISA against the spike antigen, but samples were selected blinded from these results.Power AnalysisIn addition, our sample size limited our power to detect clinically meaningful differences in sensitivities.Sex as a biological variableRESULTS Patient Characteristics Of the 36 individuals with SARS-CoV-2 infection included in the study, mean age 43 (±13) years and 19 were female, 16 were male, and 1 did not have information available on gender.

    Table 2: Resources

    Antibodies
    SentencesResources
    ABSTRACT Background: Several serological assays have been developed to detect anti-SARS-CoV-2 IgG antibodies, but evidence about their comparative performance is limited.
    anti-SARS-CoV-2 IgG
    suggested: None
    The 3 different FDA EUA SARS-CoV-2 IgG assays and the Yale anti-spike ELISA used in this study detect antibodies directed against different immunogenic components of SARS- CoV-2.
    anti-spike ELISA
    suggested: None
    Nevertheless, this study provides important information on the correlations between 3 commercially available serology tests, as well as the Yale anti-spike ELISA, and also information on the sensitivity of these tests, which can help guide future research on the seroprevalence of IgG antibodies to SARS-CoV-2.
    anti-spike ELISA,
    suggested: None
          <div style="margin-bottom:8px">
        <div><b>SARS-CoV-2</b></div>
        <div>suggested: None</div>
      </div>
    </td></tr><tr><td style="min-width:100px;text-align:center; padding-top:4px;" colspan="2"><b>Software and Algorithms</b></td></tr><tr><td style="min-width:100px;text=align:center"><i>Sentences</i></td><td style="min-width:100px;text-align:center"><i>Resources</i></td></tr><tr><td style="min-width:100px;vertical-align:top;border-bottom:1px solid lightgray">The sensitivities of the tests were not significantly different (Abbott Architect, Ortho Vitros, Euroimmmun, and Yale assays: 86% (95% confidence interval [CI], 71– 95), 94% (95% CI, 81–99), 86% (95% CI, 71–95), and 94% (95% CI, 81–99), respectively; pvalue=0.464).</td><td style="min-width:100px;border-bottom:1px solid lightgray">
      <div style="margin-bottom:8px">
        <div><b>Abbott Architect</b></div>
        <div>suggested: (Abbott ARCHITECT i1000sr System, <a href="https://scicrunch.org/resources/Any/search?q=SCR_018371">SCR_018371</a>)</div>
      </div>
    </td></tr><tr><td style="min-width:100px;vertical-align:top;border-bottom:1px solid lightgray">The cross-correlation coefficient ranged from 0.85–0.98 between three anti-spike protein assa (Ortho Vitros, Euroimmun, Yale) and was 0.58–0.71 between the three anti-spike protein assa and the anti-nucleocapsid assay (Abbott).</td><td style="min-width:100px;border-bottom:1px solid lightgray">
      <div style="margin-bottom:8px">
        <div><b>Abbott</b></div>
        <div>suggested: (Abbott, <a href="https://scicrunch.org/resources/Any/search?q=SCR_010477">SCR_010477</a>)</div>
      </div>
    </td></tr><tr><td style="min-width:100px;vertical-align:top;border-bottom:1px solid lightgray">The following cutoffs were used for reporting of results according to the manufacturer’s instructions: 1) Abbott Architect IgG assay3: <1.4 = negative and ≥1.4 = positiv 2) Ortho Vitros IgG assay4: <1.00 = negative and ≥1.00 = positive; and 3) Euroimmun IgG assay5: <0.8 = negative, ≥0.8 to <1.1 = equivocal, and ≥1.1 = positive.</td><td style="min-width:100px;border-bottom:1px solid lightgray">
      <div style="margin-bottom:8px">
        <div><b>Abbott Architect IgG</b></div>
        <div>suggested: (Resource Identification Portal, <a href="https://scicrunch.org/resources/Any/search?q=SCR_004098">SCR_004098</a>)</div>
      </div>
    </td></tr><tr><td style="min-width:100px;vertical-align:top;border-bottom:1px solid lightgray">Krumholz works under contract with the Centers for Medicare & Medicaid Services to support quality measurement programs; was a recipient of a research grant, throug Yale, from Medtronic and the United States Food and Drug Administration to develop methods for post-market surveillance of medical devices; was a recipient of a research grant with Medtronic and is the recipient of a research grant from Johnson & Johnson, through Yale University, to support clinical trial data sharing; was a recipient of a research agreement, through Yale University, from the Shenzhen Center for Health Information for work to advance intelligent disease prevention and health promotion; collaborates with the National Center for Cardiovascular Diseases in Beijing; receives payment from the Arnold & Porter Law Firm for work related to the Sanofi clopidogrel litigation, from the Ben C.</td><td style="min-width:100px;border-bottom:1px solid lightgray">
      <div style="margin-bottom:8px">
        <div><b>Medtronic</b></div>
        <div>suggested: (Medtronic, <a href="https://scicrunch.org/resources/Any/search?q=SCR_003988">SCR_003988</a>)</div>
      </div>
    </td></tr></table>

    Data from additional tools added to each annotation on a weekly basis.

    About SciScore

    SciScore is an automated tool that is designed to assist expert reviewers by finding and presenting formulaic information scattered throughout a paper in a standard, easy to digest format. SciScore is not a substitute for expert review. SciScore checks for the presence and correctness of RRIDs (research resource identifiers) in the manuscript, and detects sentences that appear to be missing RRIDs. SciScore also checks to make sure that rigor criteria are addressed by authors. It does this by detecting sentences that discuss criteria such as blinding or power analysis. SciScore does not guarantee that the rigor criteria that it detects are appropriate for the particular study. Instead it assists authors, editors, and reviewers by drawing attention to sections of the manuscript that contain or should contain various rigor criteria and key resources. For details on the results shown here, including references cited, please follow this link.

  3. Version published to 10.1101/2020.07.29.20164343 on medRxiv