Evaluation of the accuracy, ease of use and limit of detection of novel, rapid, antigen-detecting point-of-care diagnostics for SARS-CoV-2

  1. L.J. Krüger
  2. M. Gaeddert
  3. L. Köppel
  4. L. E. Brümmer
  5. C. Gottschalk
  6. I.B. Miranda
  7. P. Schnitzler
  8. H.G. Kräusslich
  9. A.K. Lindner
  10. O. Nikolai
  11. F.P. Mockenhaupt
  12. J. Seybold
  13. V.M. Corman
  14. C. Drosten
  15. N.R. Pollock
  16. A.I. Cubas-Atienzar
  17. K. Kontogianni
  18. A. Collins
  19. A. H. Wright
  20. B. Knorr
  21. A. Welker
  22. M. de Vos
  23. J.A. Sacks
  24. E.R. Adams
  25. C.M. Denkinger
  26. for the study team

Reviewed by

Read the full article See related articles

Listed in

Log in to save this article

Abstract

Background

Reliable point-of-care (POC) diagnostics not requiring laboratory infrastructure could be a game changer in the COVID-19 pandemic, particularly in the Global South. We assessed performance, limit of detection and ease-of-use of three antigen-detecting, rapid POC tests (Ag-RDT) for SARS-CoV-2 .

Methods

This prospective, multi-centre diagnostic accuracy study recruited participants suspected to have SARS-CoV-2 in Germany and the UK. Paired nasopharyngeal swabs (NP) or NP and/or oropharyngeal swabs (OP) were collected from participants (one for clinical RT-PCR and one for Ag-RDT). Performance of each of three Ag-RDTs was compared to RT-PCR overall, and according to predefined subcategories e.g. cycle threshold (CT)-value, days from symptoms onset, etc. In addition, limited verification of the analytical limit-of-detection (LOD) was determined. To understand the usability a System Usability Scale (SUS) questionnaire and ease-of-use (EoU) assessment were performed.

Results

Between April 17 th and August 25 th , 2020, 2417 participants were enrolled, with 70 (3.0%) testing positive by RT-PCR. The best-performing test (SD Biosensor, Inc. STANDARD Q) was 76×6% (95% Confidence Interval (CI) 62×8-86×4) sensitive and 99×3% (CI 98×6-99×6) specific. A sub-analysis showed all samples with RT-PCR CT-values <25 were detectable by STANDARD Q. The test was considered easy-to-use (SUS 86/100) and suitable for POC. Bioeasy and Coris showed specificity of 93×1% (CI 91×0%-94×8%) and 95×8% (CI 93×4%-97×4%), respectively, not meeting the predefined target of ≥98%.

Conclusion

There is large variability in performance of Ag-RDT with SD Biosensor showing promise. Given the usability at POC, this test is likely to have impact despite imperfect sensitivity; however further research and modelling are needed.

Article activity feed

  1. ScreenIT

    SciScore for 10.1101/2020.10.01.20203836: (What is this?)

    Please note, not all rigor criteria are appropriate for all manuscripts.

    Table 1: Rigor

    Institutional Review Board StatementIRB: Ethic statement: The study protocol was approved by the ethical review committee at the Heidelberg University Hospital for the study sites in Heidelberg and Berlin (Registration number S-180/2020), and by the NHS review board
    Consent: Each participant provided written informed consent.
    Randomizationnot detected.
    BlindingCoris and SD Biosensor tests were read by two operators, each blinded to the result of the other.
    Power Analysisnot detected.
    Sex as a biological variablenot detected.
    Cell Line Authenticationnot detected.

    Table 2: Resources

    Experimental Models: Cell Lines
    SentencesResources
    The SARS-CoV-2 isolate REMRQ0001/Human/2020/Liverpool was propagated in Vero E6 cells (C1008; African green monkey kidney cells).
    Vero E6
    suggested: RRID:CVCL_XD71)
    Software and Algorithms
    SentencesResources
    Both SUS and Ease of Use assessment were analysed with Microsoft Excel.
    Microsoft Excel
    suggested: (Microsoft Excel, RRID:SCR_016137)

    Results from OddPub: We did not detect open data. We also did not detect open code. Researchers are encouraged to share open data when possible (see Nature blog).

    Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:
    Furthermore, stability only up to 30° Celsius and required storage conditions are a limitation for the use in the Global South, and should be considered carefully in such settings. Our study also shows wide variability in performance between Ag-RDTs with respect to performance and ease-of-use, indicating the importance of manufacturer-independent validations. While our findings largely corroborate the performance reported in the IFU of SD Biosensor, for Coris and Bioeasy, they differed substantially in respect to specificity. For SD Biosensor, the lower overall performance compared to the results in the IFU could relate to the fact that for some patients in our study OP swabs were used. While this is not recommended by the manufacturer, the use of OP swabs when NP is contraindicated, reflects a more clinically realistic scenario. NP/OP swabs were used based on institutional recommendations in Berlin and Liverpool. While more proteases are expected in the saliva that can degrade antigen, there should not be an impact on the sensitivity of Ag-RDTs in our study, as swabs were processed immediately. Another aspect that could explain a lower senstivtiy is the population tested in the study. The population tested was likely representative of the spectrum of COVID-19, with less ill patients likely to have lower viral load at diagnosis and more likely Ag-RDT negative. A higher clinical sensitivity would be expected when only hospitalized patients were tested.30 For Bioeasy, our point...

    Results from TrialIdentifier: No clinical trial numbers were referenced.

    Results from Barzooka: We did not find any issues relating to the usage of bar graphs.

    Results from JetFighter: We did not find any issues relating to colormaps.

    Results from rtransparent:
    • Thank you for including a conflict of interest statement. Authors are encouraged to include this statement when submitting to a journal.
    • Thank you for including a funding statement. Authors are encouraged to include this statement when submitting to a journal.
    • No protocol registration statement was detected.

    About SciScore

    SciScore is an automated tool that is designed to assist expert reviewers by finding and presenting formulaic information scattered throughout a paper in a standard, easy to digest format. SciScore checks for the presence and correctness of RRIDs (research resource identifiers), and for rigor criteria such as sex and investigator blinding. For details on the theoretical underpinning of rigor criteria and the tools shown here, including references cited, please follow this link.

  2. Version published to 10.1101/2020.10.01.20203836 on medRxiv