Diagnostic Accuracy of FDA Authorized Serology Tests to Detect SARS-CoV-2 Antibodies: A Systematic Review and Meta-analysis

  1. Rajesh Pandey
  2. Anand Gourishankar

Reviewed by

Read the full article See related articles

Listed in

Log in to save this article

Abstract

Importance

Serology tests are diagnostic and complementary to molecular tests during the COVID-19 pandemic.

Objective

To evaluate the diagnostic accuracy of FDA authorized serology tests for the detection of SARS-CoV-2 infection.

Data sources

A search of MEDLINE, SCOPUS, CINAHL Plus, and EMBASE up to April 4, 2020, was performed to identify studies using the “COVID 19 testing” and “meta-analysis.” FDA website was accessed for the list of tests for emergency use authorization (EUA).

Study Selection

Manufacturer reported serology tests published in the FDA website were selected. Two reviewers independently assessed the eligibility of the selected reports.

Data extraction and synthesis

The meta-analysis was performed in accordance with the PRISMA guidelines. A bivariate analysis using the “random-effects model” was applied for pooled summary estimates of sensitivity, specificity, and the summary receiver operating characteristic curves.

Main outcomes and measures

The primary outcome was the diagnostic accuracy of the serology test for detecting SARS-CoV-2 infection. Subgroup analysis of the diagnostic accuracy with lag time between symptom onset and testing were studied.

Results

Seven manufacturer listed reports were included. The pooled sensitivity was 87% (95% CI, 78% - 93%), the pooled specificity was 100% (95% CI, 97% - 100%), and the area under the hierarchical summary receiver operating characteristic curve was 0.97. At ≤ 7 days, sensitivity was 44% (95% CI, 21% - 70%), and for 8-14 days, sensitivity was 84% (95% CI, 67 % - 94%).For blood draws ≥ 15 days after the onset of symptoms, sensitivity was 96% (95% CI, 93% - 98%). Heterogeneity was substantial, and the risk of bias was low in this analysis.

Conclusions and relevance

FDA authorized serology tests demonstrate high diagnostic accuracy for SARS-CoV-2 infection (certainty of evidence: moderate). There is a wide variation in the test accuracy based on the duration between the onset of symptoms and the tests (certainty of evidence: low).

Key– points

Questions

What is the pooled diagnostic accuracy of FDA authorized serology tests to detect SARS-CoV-2 antibodies?

Findings

In this systematic review and meta-analysis of seven reports from FDA authorized serology tests to detect antibodies against SARS-CoV2 antibodies (3336 patients/ samples) pooled sensitivity was 87%, and pooled specificity was almost 100%. There was a wide variation in test performance based on the duration between the onset of symptoms and the tests.

Meaning

FDA authorized tests are highly accurate to detect antibodies against SARS-CoV-2 antibodies if tests are performed under a similar condition, as presented in the original report. There is a wide variation in the test performance based on the time interval between the onset of symptoms to the tests.

Article activity feed

  1. ScreenIT

    SciScore for 10.1101/2020.10.07.20208553: (What is this?)

    Please note, not all rigor criteria are appropriate for all manuscripts.

    Table 1: Rigor

    Institutional Review Board Statementnot detected.
    Randomizationnot detected.
    Blindingnot detected.
    Power Analysisnot detected.
    Sex as a biological variablenot detected.

    Table 2: Resources

    Antibodies
    SentencesResources
    11 Inclusions also had the following criteria: index test must include qualitative detection of anti-SARS-CoV-2 antibodies [Immunoglobulin G (IgG) or Immunoglobulin M (IgM), or both], and target condition must be confirmed by RT-PCR (the reference standard).
    anti-SARS-CoV-2
    suggested: None
    Immunoglobulin M (IgM)
    suggested: None
    Software and Algorithms
    SentencesResources
    We performed a search for systematic reviews up to April 4, 2020, using the term “COVID 19 testing” and “meta-analysis” in the following databases: MEDLINE, CINAHL Plus, EMBASE, and SCOPUS.
    MEDLINE
    suggested: (MEDLINE, RRID:SCR_002185)
    EMBASE
    suggested: (EMBASE, RRID:SCR_001650)
    We assessed the methodological quality of each study by following the guidelines from the Cochrane Screening and Diagnostic Test Methods Group, a tool adapted from the QUADAS-2.
    Cochrane Screening
    suggested: (Robot Reviewer, RRID:SCR_018961)
    We performed data analysis using methods described in the Cochrane Handbook of Diagnostic Test Accuracy (DTA) Reviews. 17 We created forest plots with 95% confidence intervals (CI) for sensitivity and specificity for each study using Review Manager 5.
    Cochrane Handbook
    suggested: None

    Results from OddPub: We did not detect open data. We also did not detect open code. Researchers are encouraged to share open data when possible (see Nature blog).

    Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:
    These cross-reactivity limitations should be considered while interpreting positive tests. Limitations: Information on the study population is not available to critically appraise individual reports. We could not verify the spectrum bias. Per protocol, we limited our search to FDA authorized serology tests for detecting antibodies against SARS-CoV-2. None of the serology tests have been independently verified for accuracy by the FDA. Five reports used detection of IgG or IgM or both, and the remaining two used detection of IgG only for the diagnosis of SARS-CoV-2. We accepted the threshold detection of antibodies as defined by individual reports. Heterogeneity could not be explained by threshold as the serology tests were qualitative. Heterogeneity was substantial. We could not perform meta-regression or sensitivity analysis as limited information was available. Higher heterogeneity in this review can arise from chance, patient selection, and the type of diagnostic test. We found a significant area under the curve (AUC), which represents the global summary performance of the test. The symmetric shoulder of SROC represents the variability in the studies and the trade-off between sensitivity and specificity.

    Results from TrialIdentifier: No clinical trial numbers were referenced.

    Results from Barzooka: We did not find any issues relating to the usage of bar graphs.

    Results from JetFighter: We did not find any issues relating to colormaps.

    Results from rtransparent:
    • Thank you for including a conflict of interest statement. Authors are encouraged to include this statement when submitting to a journal.
    • Thank you for including a funding statement. Authors are encouraged to include this statement when submitting to a journal.
    • No protocol registration statement was detected.

    About SciScore

    SciScore is an automated tool that is designed to assist expert reviewers by finding and presenting formulaic information scattered throughout a paper in a standard, easy to digest format. SciScore checks for the presence and correctness of RRIDs (research resource identifiers), and for rigor criteria such as sex and investigator blinding. For details on the theoretical underpinning of rigor criteria and the tools shown here, including references cited, please follow this link.

  2. Version published to 10.1101/2020.10.07.20208553v1 on medRxiv