Meta-analysis of the clinical performance of commercial SARS-CoV-2 nucleic acid and antibody tests up to 22 August 2020
This article has been Reviewed by the following groups
Listed in
- Evaluated articles (ScreenIT)
Abstract
Reliable testing for SARS-CoV-2 is key for the management of the COVID-19 pandemic.
Aim
We estimate diagnostic accuracy for nucleic acid and antibody tests 5 months into the COVID-19 pandemic, and compare with manufacturer-reported accuracy.
Methods
We reviewed the clinical performance of SARS-CoV-2 nucleic acid and antibody tests based on 93,757 test results from 151 published studies and 20,205 new test results from 12 countries in the European Union and European Economic Area (EU/EEA).
Results
Pooling the results and considering only results with 95% confidence interval width ≤ 5%, we found four nucleic acid tests, including one point-of-care test and three antibody tests, with a clinical sensitivity ≥ 95% for at least one target population (hospitalised, mild or asymptomatic, or unknown). Nine nucleic acid tests and 25 antibody tests, 12 of them point-of-care tests, had a clinical specificity of ≥ 98%. Three antibody tests achieved both thresholds. Evidence for nucleic acid point-of-care tests remains scarce at present, and sensitivity varied substantially. Study heterogeneity was low for eight of 14 sensitivity and 68 of 84 specificity results with confidence interval width ≤ 5%, and lower for nucleic acid tests than antibody tests. Manufacturer-reported clinical performance was significantly higher than independently assessed in 11 of 32 and four of 34 cases, respectively, for sensitivity and specificity, indicating a need for improvement in this area.
Conclusion
Continuous monitoring of clinical performance within more clearly defined target populations is needed.
Article activity feed
-
-
SciScore for 10.1101/2020.09.16.20195917: (What is this?)
Please note, not all rigor criteria are appropriate for all manuscripts.
Table 1: Rigor
Institutional Review Board Statement not detected. Randomization not detected. Blinding Studies with an ineligible design, such as blinded tests, analytical validation only, use of another threshold for positivity than in the instructions for use, comparisons between different specimen types or use of an antibody rather than nucleic acid test as reference test for any type of index test were subsequently excluded as well. Power Analysis not detected. Sex as a biological variable not detected. Table 2: Resources
Antibodies Sentences Resources Search strategy and selection criteria: Studies containing potentially usable data on clinical performance of SARS-CoV-2 nucleic acid, antigen and antibody tests were … SciScore for 10.1101/2020.09.16.20195917: (What is this?)
Please note, not all rigor criteria are appropriate for all manuscripts.
Table 1: Rigor
Institutional Review Board Statement not detected. Randomization not detected. Blinding Studies with an ineligible design, such as blinded tests, analytical validation only, use of another threshold for positivity than in the instructions for use, comparisons between different specimen types or use of an antibody rather than nucleic acid test as reference test for any type of index test were subsequently excluded as well. Power Analysis not detected. Sex as a biological variable not detected. Table 2: Resources
Antibodies Sentences Resources Search strategy and selection criteria: Studies containing potentially usable data on clinical performance of SARS-CoV-2 nucleic acid, antigen and antibody tests were first extracted from systematic reviews on this topic. SARS-CoV-2 nucleic acid, antigensuggested: NoneSoftware and Algorithms Sentences Resources These reviews were identified through an initial Pubmed (Medline) search for systematic reviews and meta-analysis for ‘COVID-19’ and ‘SARS-CoV-2’, followed by snowballing using the ‘find similar articles’ feature. Medlinesuggested: (MEDLINE, RRID:SCR_002185)5] Finally, Pubmed was searched according to the query in supplementary Figure S1. Pubmedsuggested: (PubMed, RRID:SCR_004846)Results from OddPub: We did not detect open data. We also did not detect open code. Researchers are encouraged to share open data when possible (see Nature blog).
Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:Limitations of this paper include that most of the included studies have a substantial risk of bias in the sample selection, especially for the sensitivity panel, as established also in the assessments performed in the systematic reviews that were used as a source. Results were mainly based on hospitalised cases or poorly defined populations, whereas the population of interest consists often of symptomatic cases in general or even asymptomatic cases, and differences in performance may exist depending on disease severity. While this review addresses a pressing need for actionable clinical performance data, ideally, the clinical performance should be assessed through prospective studies or clinical trials with a guaranteed unbiased sample selection for a clearly defined target population and intended use of the test. Given the difficulty of assessing and extracting the data from individual studies in a coherent way, we recommend that the Standard for Reporting of Diagnostic Accuracy Studies (STARD) should also be followed when publishing the results.[25] In this context, the selection of the reference test is particularly important with respect to reference negative samples. As described in some of the assessed studies, it should be avoided that index test results are considered as false positives while the samples are from actual cases and for this reason we excluded nucleic acid negative samples from suspected COVID-19 patients altogether. We expect therefore little bias in t...
Results from TrialIdentifier: No clinical trial numbers were referenced.
Results from Barzooka: We did not find any issues relating to the usage of bar graphs.
Results from JetFighter: We did not find any issues relating to colormaps.
Results from rtransparent:- Thank you for including a conflict of interest statement. Authors are encouraged to include this statement when submitting to a journal.
- Thank you for including a funding statement. Authors are encouraged to include this statement when submitting to a journal.
- No protocol registration statement was detected.
-
