Evaluation of Serological Tests for SARS-CoV-2: Implications for Serology Testing in a Low-Prevalence Setting

  1. Katherine Bond
  2. Suellen Nicholson
  3. Seok Ming Lim
  4. Theo Karapanagiotidis
  5. Eloise Williams
  6. Douglas Johnson
  7. Tuyet Hoang
  8. Cheryll Sia
  9. Damian Purcell
  10. Francesca Mordant
  11. Sharon R Lewin
  12. Mike Catton
  13. Kanta Subbarao
  14. Benjamin P Howden
  15. Deborah A Williamson

Reviewed by

Read the full article See related articles

Listed in

Log in to save this article

Abstract

Background

Robust serological assays are essential for long-term control of the COVID-19 pandemic. Many recently released point-of-care (PoCT) serological assays have been distributed with little premarket validation.

Methods

Performance characteristics for 5 PoCT lateral flow devices approved for use in Australia were compared to a commercial enzyme immunoassay (ELISA) and a recently described novel surrogate virus neutralization test (sVNT).

Results

Sensitivities for PoCT ranged from 51.8% (95% confidence interval [CI], 43.1%–60.4%) to 67.9% (95% CI, 59.4%–75.6%), and specificities from 95.6% (95% CI, 89.2%–98.8%) to 100.0% (95% CI, 96.1%–100.0%). ELISA sensitivity for IgA or IgG detection was 67.9% (95% CI, 59.4%–75.6%), increasing to 93.8% (95% CI, 85.0%–98.3%) for samples >14 days post symptom onset. sVNT sensitivity was 60.9% (95% CI, 53.2%–68.4%), rising to 91.2% (95% CI, 81.8%–96.7%) for samples >14 days post symptom onset, with specificity 94.4% (95% CI, 89.2%–97.5%).

Conclusions

Performance characteristics for COVID-19 serological assays were generally lower than those reported by manufacturers. Timing of specimen collection relative to onset of illness or infection is crucial in reporting of performance characteristics for COVID-19 serological assays. The optimal algorithm for implementing serological testing for COVID-19 remains to be determined, particularly in low-prevalence settings.

Article activity feed

  1. Version published to 10.1093/infdis/jiaa467
  2. ScreenIT

    SciScore for 10.1101/2020.05.31.20118273: (What is this?)

    Please note, not all rigor criteria are appropriate for all manuscripts.

    Table 1: Rigor

    Institutional Review Board StatementIRB: Ethics: Ethical approval for this project was obtained from the Melbourne Health Human Research Ethics Committee (RMH HREC QA2020052).
    Randomizationnot detected.
    Blindingnot detected.
    Power Analysisnot detected.
    Sex as a biological variablenot detected.

    Table 2: Resources

    Antibodies
    SentencesResources
    The ELISA involves semi-quantitative detection of anti-SARS-CoV-2 IgA or IgG antibodies in serum through binding to a recombinant structural antigen (S1 domain of the Spike protein) fixed to reagent wells.
    anti-SARS-CoV-2 IgA
    suggested: None
    IgG antibodies
    suggested: None
    If test sera contain anti-SARS-CoV-2 antibodies, a second incubation step using enzyme-labelled anti-human IgA or anti-human IgG will catalyse a colour reaction, detected by an optical density reader.
    anti-SARS-CoV-2
    suggested: None
    anti-human IgA
    suggested: None
    anti-human IgG
    suggested: None
    Software and Algorithms
    SentencesResources
    Receiver operating characteristic (ROC) area under the curve (AUC) analysis was performed in GraphPad Prism (version 8.4.2).
    GraphPad Prism
    suggested: (GraphPad Prism, RRID:SCR_002798)

    Results from OddPub: We did not detect open data. We also did not detect open code. Researchers are encouraged to share open data when possible (see Nature blog).

    Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:
    However, in order to appropriately deploy serological testing, it is critical to understand the limitations of test performance in the epidemiological context in which tests are used. This is particularly important in a setting such as Australia, which, based on the number of reported cases of COVID-19 (7,060 cases as of 18th May, 2020), has an estimated COVID-19 period prevalence of 0.03% (16). As such, even with highly sensitive and specific serological tests, the majority of positive results are likely to represent false positives. When considering the use of serology to inform policies relating to relaxing of physical distancing interventions, specificity of the assay becomes critical. If the majority of those identified as immune are actually false positive results, then the threshold to maintain immunity within the community will not be achieved (17). Analogous to HIV testing in low-prevalence settings (18), serological testing for SARS-CoV-2 may require a ‘two-step’ approach, whereby a sensitive high-throughput ‘screening’ assay is followed by a high specificity assay for confirmation (e.g. neutralisation testing or Western blot). This approach could facilitate seroepidemiological studies in low-prevalence settings, which are required to better understand the extent of COVID-19 infection at a population-level. Ongoing questions remain however, about the duration and type of antibody response to SARS-CoV-2, particularly around the protective effect of neutralising antib...

    Results from TrialIdentifier: No clinical trial numbers were referenced.

    Results from Barzooka: We did not find any issues relating to the usage of bar graphs.

    Results from JetFighter: We did not find any issues relating to colormaps.

    Results from rtransparent:
    • Thank you for including a conflict of interest statement. Authors are encouraged to include this statement when submitting to a journal.
    • Thank you for including a funding statement. Authors are encouraged to include this statement when submitting to a journal.
    • No protocol registration statement was detected.

    About SciScore

    SciScore is an automated tool that is designed to assist expert reviewers by finding and presenting formulaic information scattered throughout a paper in a standard, easy to digest format. SciScore checks for the presence and correctness of RRIDs (research resource identifiers), and for rigor criteria such as sex and investigator blinding. For details on the theoretical underpinning of rigor criteria and the tools shown here, including references cited, please follow this link.

  3. Version published to 10.1101/2020.05.31.20118273v1 on medRxiv