Multisite Clinical Validation of Isothermal Amplification-Based SARS-CoV-2 Detection Assays Using Different Sampling Strategies

  1. Kanan T. Desai
  2. Karla Alfaro
  3. Laura Mendoza
  4. Matthew Faron
  5. Brian Mesich
  6. Mauricio Maza
  7. Rhina Dominguez
  8. Adriana Valenzuela
  9. Chyntia Díaz Acosta
  10. Magaly Martínez
  11. Juan C. Felix
  12. Rachel Masch
  13. Jennifer S. Smith
  14. Sofia Gabrilovich
  15. Tracy Wu
  16. Michael Plump
  17. Akiva P. Novetsky
  18. Mark H. Einstein
  19. Nataki C. Douglas
  20. Miriam Cremer
  21. Nicolas Wentzensen

Reviewed by

Read the full article See related articles

Listed in

Log in to save this article

Abstract

Numerous SARS-CoV-2 detection assays have been developed and introduced into the market under emergency use authorizations (EUAs). EUAs are granted primarily based on small studies of analytic sensitivity and specificity with limited clinical validations.

Article activity feed

  1. Version published to 10.1128/spectrum.00846-21
  2. ScreenIT

    SciScore for 10.1101/2021.07.01.21259879: (What is this?)

    Please note, not all rigor criteria are appropriate for all manuscripts.

    Table 1: Rigor

    EthicsIRB: The study protocol at all the sites was approved by the respective local institutional ethical review boards.
    Field Sample Permit: History of symptoms was collected from the subjects at the time of sample collection.
    Sex as a biological variablenot detected.
    Randomizationnot detected.
    Blindingnot detected.
    Power Analysisnot detected.

    Table 2: Resources

    Software and Algorithms
    SentencesResources
    The Genie® software automatically analyzed the individual sample test results as positive or negative based on the amplification plot and annealing temperature.
    Genie®
    suggested: (GENIE, RRID:SCR_009197)
    Data analysis was performed using IBM® SPSS software.
    SPSS
    suggested: (SPSS, RRID:SCR_002865)

    Results from OddPub: We did not detect open data. We also did not detect open code. Researchers are encouraged to share open data when possible (see Nature blog).

    Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:
    It demonstrates the limitations of EUAs which may not necessarily translate to acceptable clinical performance for all tests in all settings. A thorough clinical validation of diagnostic assays on a standardized panel of samples in clinical settings is advisable before its widespread adoption for clinical use. We do not fully understand the reason for the variation in test performance across study-sites. Importantly, the populations at each site was different with respect to SARS-CoV-2 prevalence, clinical symptoms, and other factors, but stratified analyses showed similar performance at all sites except for El Salvador. Given that invalid results were rare and did not differ across the study sites and there was no consistent pattern observed in ct-values for the internal control [mean ct-values for the internal control for NP: 28.4 (95% CI: 28.1-28.7) (El Salvador), 34.1 (95% CI: 32.8-35.1) (MCW), 22.7 (21.9-23.4) (Paraguay), 25.4 (23.4-27.3) (NJMS); for saliva: 19.4 (95% CI: 19.1-19.7), 23.6 (95% CI: 23.1-24.2) (MCW), 22.1 (21.3-22.8) (Paraguay)], we do not attribute the lower sensitivity in our validation in El Salvador to sampling variation. Rather we hypothesize the lower sensitivity of the Atila iAMP test in El Salvador to be related to multiple factors: relatively higher proportion of asymptomatic subjects as compared to Paraguay (69.8% versus 21.9%) and operator-dependent nature of the assay due to the hands-on nature of the test to set up the reaction (27). However, ...

    Results from TrialIdentifier: No clinical trial numbers were referenced.

    Results from Barzooka: We did not find any issues relating to the usage of bar graphs.

    Results from JetFighter: We did not find any issues relating to colormaps.

    Results from rtransparent:
    • Thank you for including a conflict of interest statement. Authors are encouraged to include this statement when submitting to a journal.
    • Thank you for including a funding statement. Authors are encouraged to include this statement when submitting to a journal.
    • No protocol registration statement was detected.

    Results from scite Reference Check: We found no unreliable references.

    About SciScore

    SciScore is an automated tool that is designed to assist expert reviewers by finding and presenting formulaic information scattered throughout a paper in a standard, easy to digest format. SciScore checks for the presence and correctness of RRIDs (research resource identifiers), and for rigor criteria such as sex and investigator blinding. For details on the theoretical underpinning of rigor criteria and the tools shown here, including references cited, please follow this link.

  3. Version published to 10.1101/2021.07.01.21259879v1 on medRxiv