Effectiveness of tests to detect the presence of SARS-CoV-2 virus, and antibodies to SARS-CoV-2, to inform COVID-19 diagnosis: a rapid systematic review

  1. David Jarrom
  2. Lauren Elston
  3. Jennifer Washington
  4. Matthew Prettyjohns
  5. Kimberley Cann
  6. Susan Myles
  7. Peter Groves

Reviewed by

Read the full article

Listed in

Log in to save this article

Abstract

We undertook a rapid systematic review with the aim of identifying evidence that could be used to answer the following research questions: (1) What is the clinical effectiveness of tests that detect the presence of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) to inform COVID-19 diagnosis? (2) What is the clinical effectiveness of tests that detect the presence of antibodies to the SARS-CoV-2 virus to inform COVID-19 diagnosis?

Design and setting

Systematic review and meta-analysis of studies of diagnostic test accuracy. We systematically searched for all published evidence on the effectiveness of tests for the presence of SARS-CoV-2 virus, or antibodies to SARS-CoV-2, up to 4 May 2020, and assessed relevant studies for risks of bias using the QUADAS-2 framework.

Main outcome measures

Measures of diagnostic accuracy (sensitivity, specificity, positive/negative predictive value) were the main outcomes of interest. We also included studies that reported influence of testing on subsequent patient management, and that reported virus/antibody detection rates where these facilitated comparisons of testing in different settings, different populations or using different sampling methods.

Results

38 studies on SARS-CoV-2 virus testing and 25 studies on SARS-CoV-2 antibody testing were identified. We identified high or unclear risks of bias in the majority of studies, most commonly as a result of unclear methods of patient selection and test conduct, or because of the use of a reference standard that may not definitively diagnose COVID-19. The majority were in hospital settings, in patients with confirmed or suspected COVID-19 infection. Pooled analysis of 16 studies (3818 patients) estimated a sensitivity of 87.8% (95% CI 81.5% to 92.2%) for an initial reverse-transcriptase PCR test. For antibody tests, 10 studies reported diagnostic accuracy outcomes: sensitivity ranged from 18.4% to 96.1% and specificity 88.9% to 100%. However, the lack of a true reference standard for SARS-CoV-2 diagnosis makes it challenging to assess the true diagnostic accuracy of these tests. Eighteen studies reporting different sampling methods suggest that for virus tests, the type of sample obtained/type of tissue sampled could influence test accuracy. Finally, we searched for, but did not identify, any evidence on how any test influences subsequent patient management.

Conclusions

Evidence is rapidly emerging on the effectiveness of tests for COVID-19 diagnosis and management, but important uncertainties about their effectiveness and most appropriate application remain. Estimates of diagnostic accuracy should be interpreted bearing in mind the absence of a definitive reference standard to diagnose or rule out COVID-19 infection. More evidence is needed about the effectiveness of testing outside of hospital settings and in mild or asymptomatic cases. Implementation of public health strategies centred on COVID-19 testing provides opportunities to explore these important areas of research.

Article activity feed

  1. Version published to 10.1136/bmjebm-2020-111511
  2. ScreenIT

    SciScore for 10.1101/2020.08.10.20171777: (What is this?)

    Please note, not all rigor criteria are appropriate for all manuscripts.

    Table 1: Rigor

    Institutional Review Board Statementnot detected.
    Randomizationnot detected.
    Blindingnot detected.
    Power Analysisnot detected.
    Sex as a biological variablenot detected.

    Table 2: Resources

    Software and Algorithms
    SentencesResources
    The databases searched were Medline, Embase, Cochrane Library, International Network of Agencies for Health Technology Assessment (INAHTA)HTA database & Open Grey, to include all evidence published up to 4 May 2020.
    Medline
    suggested: (MEDLINE, RRID:SCR_002185)
    Embase
    suggested: (EMBASE, RRID:SCR_001650)
    Cochrane Library
    suggested: (Cochrane Library, RRID:SCR_013000)
    Pooled estimates were calculated for diagnostic accuracy outcomes using a random effects bivariate binomial model in MetaDTA v1.25.[
    MetaDTA
    suggested: None

    Results from OddPub: We did not detect open data. We also did not detect open code. Researchers are encouraged to share open data when possible (see Nature blog).

    Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:
    Of the 25 studies that assessed antibody tests, 10 reported diagnostic accuracy in terms of both sensitivity and specificity, almost all using RT-PCR (initial or repeat testing) as the reference standard.[18,22,23,25-27,30-33] Accepting the limitations already discussed around the absence of a diagnostic reference standard, the overall sensitivity reported in these studies varied widely, from 18.4% to 96.1% although the specificity was more consistent and ranged from 88.9% to 100%. The clinical implications of these data are that considerable uncertainty remains about the implications of a negative antibody test with a significant possibility of false negativity, while the presence of a positive antibody test carries with it a high likelihood of previous COVID-19 infection. There is very limited information available on the accuracy of point-of-care antibody tests. Our study has some limitations, primarily due to the nature of the evidence found by our searches. The rapid nature of this work (to help inform decision makers at the outset of the COVID-19 pandemic in the United Kingdom) meant some steps in a full systematic review were not completed: there was minimal consultation with decision makers on the inclusion and exclusion criteria for the review, and we did not publish our protocol in advance of commencing the review. Other limitations relate to the nature of the evidence we found, and that this work was completed during the early stages of the COVID-19 pandemic. The l...

    Results from TrialIdentifier: No clinical trial numbers were referenced.

    Results from Barzooka: We did not find any issues relating to the usage of bar graphs.

    Results from JetFighter: We did not find any issues relating to colormaps.

    Results from rtransparent:
    • Thank you for including a conflict of interest statement. Authors are encouraged to include this statement when submitting to a journal.
    • Thank you for including a funding statement. Authors are encouraged to include this statement when submitting to a journal.
    • No protocol registration statement was detected.

    About SciScore

    SciScore is an automated tool that is designed to assist expert reviewers by finding and presenting formulaic information scattered throughout a paper in a standard, easy to digest format. SciScore checks for the presence and correctness of RRIDs (research resource identifiers), and for rigor criteria such as sex and investigator blinding. For details on the theoretical underpinning of rigor criteria and the tools shown here, including references cited, please follow this link.

  3. Version published to 10.1101/2020.08.10.20171777v1 on medRxiv