How informative were early SARS-CoV-2 treatment and prevention trials? a longitudinal cohort analysis of trials registered on ClinicalTrials.gov

  1. Nora Hutchinson
  2. Katarzyna Klas
  3. Benjamin G. Carlisle
  4. Jonathan Kimmelman
  5. Marcin Waligora

Reviewed by

Read the full article

Listed in

Log in to save this article

Abstract

Early in the SARS-CoV-2 pandemic, commentators warned that some COVID trials were inadequately conceived, designed and reported. Here, we retrospectively assess the prevalence of informative COVID trials launched in the first 6 months of the pandemic.

Methods

Based on prespecified eligibility criteria, we created a cohort of Phase 1/2, Phase 2, Phase 2/3 and Phase 3 SARS-CoV-2 treatment and prevention efficacy trials that were initiated from 2020-01-01 to 2020-06-30 using ClinicalTrials.gov registration records. We excluded trials evaluating behavioural interventions and natural products, which are not regulated by the U.S. Food and Drug Administration (FDA). We evaluated trials on 3 criteria of informativeness: potential redundancy (comparing trial phase, type, patient-participant characteristics, treatment regimen, comparator arms and primary outcome), trials design (according to the recommendations set-out in the May 2020 FDA guidance document on SARS-CoV-2 treatment and prevention trials) and feasibility of patient-participant recruitment (based on timeliness and success of recruitment).

Results

We included all 500 eligible trials in our cohort, 58% of which were Phase 2 and 84.8% were directed towards the treatment of SARS-CoV-2. Close to one third of trials met all three criteria and were deemed informative (29.9% (95% Confidence Interval 23.7–36.9)). The proportion of potentially redundant trials in our cohort was 4.1%. Over half of the trials in our cohort (56.2%) did not meet our criteria for high quality trial design. The proportion of trials with infeasible patient-participant recruitment was 22.6%.

Conclusions

Less than one third of COVID-19 trials registered on ClinicalTrials.gov during the first six months met all three criteria for informativeness. Shortcomings in trial design, recruitment feasibility and redundancy reflect longstanding weaknesses in the clinical research enterprise that were likely amplified by the exceptional circumstances of a pandemic.

Article activity feed

  1. Version published to 10.1371/journal.pone.0262114
  2. Version published to 10.1101/2021.08.25.21262155v2 on medRxiv
  3. ScreenIT

    SciScore for 10.1101/2021.08.25.21262155: (What is this?)

    Please note, not all rigor criteria are appropriate for all manuscripts.

    Table 1: Rigor

    EthicsIRB: Our study was not subject to Institutional Review Board approval, as it relies on publicly accessible data and did not involve interaction with research participants.
    Sex as a biological variablenot detected.
    RandomizationBased on the U.S. Food & Drug Association (FDA) May 2020 guidance document for SARS-CoV-2 drug and biological treatment and prevention trials,14 we considered a trial to be well-designed if it was randomized, placebo-controlled (with appropriate standard of care in all arms), double-blinded and included participants aged 60 years or over (as a proxy for an at-risk population).
    Blindingnot detected.
    Power AnalysisWe did not perform a sample size calculation, as we included all trials meeting our eligibility criteria within our designated sampling timeframe.

    Table 2: Resources

    Software and Algorithms
    SentencesResources
    Tools and data synthesis: We performed data extraction using Numbat Systematic Review Manager v. 2.11 (RRID:SCR_019207).
    Numbat
    detected: Numbat ( RRID:SCR_019207)
    16 All analyses were performed using R version 3.6.3.17 We retrieved historical versions of ClinicalTrials.gov using Clinical Trials History Scraper (RRID:SCR_019229).
    Clinical Trials History Scraper
    detected: Clinical Trials History Scraper ( RRID:SCR_019229)

    Results from OddPub: Thank you for sharing your data.

    Results from LimitationRecognizer: An explicit section about the limitations of the techniques employed in this study was not found. We encourage authors to address study limitations.

    Results from TrialIdentifier: No clinical trial numbers were referenced.

    Results from Barzooka: We did not find any issues relating to the usage of bar graphs.

    Results from JetFighter: We did not find any issues relating to colormaps.

    Results from rtransparent:
    • Thank you for including a conflict of interest statement. Authors are encouraged to include this statement when submitting to a journal.
    • Thank you for including a funding statement. Authors are encouraged to include this statement when submitting to a journal.
    • No protocol registration statement was detected.

    Results from scite Reference Check: We found no unreliable references.

    About SciScore

    SciScore is an automated tool that is designed to assist expert reviewers by finding and presenting formulaic information scattered throughout a paper in a standard, easy to digest format. SciScore checks for the presence and correctness of RRIDs (research resource identifiers), and for rigor criteria such as sex and investigator blinding. For details on the theoretical underpinning of rigor criteria and the tools shown here, including references cited, please follow this link.

  4. Version published to 10.1101/2021.08.25.21262155v1 on medRxiv