Comparison of the Accula SARS-CoV-2 Test with a Laboratory-Developed Assay for Detection of SARS-CoV-2 RNA in Clinical Nasopharyngeal Specimens

  1. Catherine A. Hogan
  2. Natasha Garamani
  3. Andrew S. Lee
  4. Jack K. Tung
  5. Malaya K. Sahoo
  6. ChunHong Huang
  7. Bryan Stevens
  8. James Zehnder
  9. Benjamin A. Pinsky

Reviewed by

Read the full article See related articles

Discuss this preprint

Start a discussion What are Sciety discussions?

Listed in

Log in to save this article

Abstract

Background

Several point-of-care (POC) molecular tests have received emergency use authorization (EUA) from the Food and Drug Administration (FDA) for diagnosis of SARS-CoV-2. The test performance characteristics of the Accula (Mesa Biotech) SARS-CoV-2 POC test need to be evaluated to inform its optimal use.

Objectives

The aim of this study was to assess test performance of the Accula SARS-CoV-2 test.

Study design

The performance of the Accula test was assessed by comparing results of 100 nasopharyngeal swab samples previously characterized by the Stanford Health Care EUA laboratory-developed test (SHC-LDT) targeting the envelope ( E ) gene. Assay concordance was assessed by overall percent agreement, positive percent agreement (PPA), negative percent agreement (NPA), and Cohen’s kappa coefficient.

Results

Overall percent agreement between the assays was 84.0% (95% confidence interval [CI] 75.3 to 90.6%), PPA was 68.0% (95% CI 53.3 to 80.5%) and the kappa coefficient was 0.68 (95% CI 0.54 to 0.82). Sixteen specimens detected by the SHC-LDT were not detected by the Accula test, and showed low viral load burden with a median cycle threshold value of 37.7. NPA was 100% (95% CI 94.2 to 100%).

Conclusion

Compared to the SHC-LDT, the Accula SARS-CoV-2 test showed excellent negative agreement. However, positive agreement was low for samples with low viral load. The false negative rate of the Accula POC test calls for a more thorough evaluation of POC test performance characteristics in clinical settings, and for confirmatory testing in individuals with moderate to high pre-test probability of SARS-CoV-2 who test negative on Accula.

Article activity feed

  1. ScreenIT

    SciScore for 10.1101/2020.05.12.092379: (What is this?)

    Please note, not all rigor criteria are appropriate for all manuscripts.

    Table 1: Rigor

    Institutional Review Board Statementnot detected.
    Randomizationnot detected.
    Blindingnot detected.
    Power Analysisnot detected.
    Sex as a biological variablenot detected.

    Table 2: Resources

    No key resources detected.

    Results from OddPub: We did not detect open data. We also did not detect open code. Researchers are encouraged to share open data when possible (see Nature blog).

    Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:
    Education of health care professionals on the limitations of SARS-CoV-2 POCT should also be implemented to ensure optimal interpretation and management of negative results. Our study has several limitations. First, NP swabs were placed in VTM or saline at the patient bedside before loading the Accula test cassette, which may have decreased sensitivity by diluting the viral inoculum. Although this is discordant with the best recommended practice by the manufacturer, it is in line with the practice at multiple institutions with clinical laboratories that have assessed SARS-CoV-2 POCT due to biosafety concerns from risk of aerosolization (8–10, 18, 19). Second, it is possible that the use of saline instead of VTM led to poorer performance of the Accula. However, aliquots from the same sample were used for parallel testing with the EUA method, which minimizes sources of variation, and represents a pragmatic comparison given widespread VTM shortages. Finally, the lateral-flow read-out of the Accula test is generally easy to interpret; however, faint lines may be more challenging to interpret and lead to result discrepancies. In summary, this study demonstrated that the Accula POCT lacks sensitivity compared to a reference EUA SARS-CoV-2 LDT. Careful consideration should be given to balance the potential advantages of rapid POCT to lower diagnostic accuracy. Individuals with moderate to high pre-test probability who initially test negative on the Accula test should undergo confirma...

    Results from TrialIdentifier: No clinical trial numbers were referenced.

    Results from Barzooka: We did not find any issues relating to the usage of bar graphs.

    Results from JetFighter: We did not find any issues relating to colormaps.

    Results from rtransparent:
    • Thank you for including a conflict of interest statement. Authors are encouraged to include this statement when submitting to a journal.
    • No funding statement was detected.
    • No protocol registration statement was detected.

    About SciScore

    SciScore is an automated tool that is designed to assist expert reviewers by finding and presenting formulaic information scattered throughout a paper in a standard, easy to digest format. SciScore checks for the presence and correctness of RRIDs (research resource identifiers), and for rigor criteria such as sex and investigator blinding. For details on the theoretical underpinning of rigor criteria and the tools shown here, including references cited, please follow this link.

  2. Version published to 10.1101/2020.05.12.092379 on bioRxiv