Analytical and Clinical Comparison of Three Nucleic Acid Amplification Tests for SARS-CoV-2 Detection

  1. Elizabeth Smith
  2. Wei Zhen
  3. Ryhana Manji
  4. Deborah Schron
  5. Scott Duong
  6. Gregory J. Berry

Reviewed by

Read the full article See related articles

Discuss this preprint

Start a discussion What are Sciety discussions?

Listed in

Log in to save this article

Abstract

Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2) was first identified in December 2019 and has quickly become a worldwide pandemic. In response, many diagnostic manufacturers have developed molecular assays for SARS-CoV-2 under the Food and Drug Administration (FDA) Emergency Use Authorization (EUA) pathway. This study compared three of these assays: the Hologic Panther Fusion SARS-CoV-2 assay (Fusion), the Hologic Aptima SARS-CoV-2 assay (Aptima) and the BioFire Diagnostics COVID-19 test (BioFire), to determine analytical and clinical performance, as well as workflow. All three assays showed a similar limit of detection (LOD) using inactivated virus, with 100% detection ranging from 500-1,000 genome equivalents/ml, whereas use of a quantified RNA transcript standard showed the same trend, but had values ranging from 62.5 to 125 copies/ml, confirming variability in absolute quantification of reference standards. The clinical correlation found that the Fusion and BioFire assays had a positive percent agreement (PPA) of 98.7%, followed by the Aptima assay at 94.7% when compared to the consensus result. All three assays exhibited 100% negative percent agreement (NPA). Analysis of discordant results revealed that all four samples missed by the Aptima assay had Ct values >37 on the Fusion assay.

In conclusion, while all three assays showed similar relative LODs, we showed differences in absolute LODs depending on which standard was employed. In addition, the Fusion and BioFire assays showed better clinical performance, while the Aptima assay showed a modest decrease in overall PPA. These findings should be kept in mind when making platform testing decisions.

Article activity feed

  1. ScreenIT

    SciScore for 10.1101/2020.05.14.097311: (What is this?)

    Please note, not all rigor criteria are appropriate for all manuscripts.

    Table 1: Rigor

    Institutional Review Board Statementnot detected.
    Randomizationnot detected.
    Blindingnot detected.
    Power Analysisnot detected.
    Sex as a biological variablenot detected.

    Table 2: Resources

    No key resources detected.

    Results from OddPub: We did not detect open data. We also did not detect open code. Researchers are encouraged to share open data when possible (see Nature blog).

    Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:
    This study does have limitations, being a single site study with a limited number of NP swab specimens (n = 150) included in the clinical evaluation. However, this specimen set was selected to be representative of our patient population (including samples from patients of all ages and both genders) and positivity rate (50-60% in the beginning of April 2020), and included samples with a range of viral loads (low, moderate, high). Some initial equivocal or invalid results (n=3) by BioFire and Fusion required retesting, but only 1 remained unresolved on retest and was removed from the agreement analysis for the Fusion assay. In conclusion, our data show that the Fusion, Aptima and BioFire SARS-CoV-2 NAATs exhibit similar LODs, even when tested using two different quantified standards. In addition, the Fusion and BioFire assays have comparable clinical performance for detection of SARS-CoV-2 in NP swabs, with a PPA of 98.7%, while the Aptima assay showed a slight reduction at 94.7% PPA. All 3 assays demonstrated 100% NPA, suggesting high specificity. These performance characteristics, as well as testing volume and workflow requirements, should be considered when making testing platform decisions.

    Results from TrialIdentifier: No clinical trial numbers were referenced.

    Results from Barzooka: We did not find any issues relating to the usage of bar graphs.

    Results from JetFighter: We did not find any issues relating to colormaps.

    Results from rtransparent:
    • No conflict of interest statement was detected. If there are no conflicts, we encourage authors to explicit state so.
    • No funding statement was detected.
    • No protocol registration statement was detected.

    About SciScore

    SciScore is an automated tool that is designed to assist expert reviewers by finding and presenting formulaic information scattered throughout a paper in a standard, easy to digest format. SciScore checks for the presence and correctness of RRIDs (research resource identifiers), and for rigor criteria such as sex and investigator blinding. For details on the theoretical underpinning of rigor criteria and the tools shown here, including references cited, please follow this link.

  2. Version published to 10.1101/2020.05.14.097311 on bioRxiv