How informative were early SARS-CoV-2 treatment and prevention trials? a longitudinal cohort analysis of trials registered on ClinicalTrials.gov
This article has been Reviewed by the following groups
Listed in
- Evaluated articles (ScreenIT)
Abstract
Early in the SARS-CoV-2 pandemic, commentators warned that some COVID trials were inadequately conceived, designed and reported. Here, we retrospectively assess the prevalence of informative COVID trials launched in the first 6 months of the pandemic.
Methods
Based on prespecified eligibility criteria, we created a cohort of Phase 1/2, Phase 2, Phase 2/3 and Phase 3 SARS-CoV-2 treatment and prevention efficacy trials that were initiated from 2020-01-01 to 2020-06-30 using ClinicalTrials.gov registration records. We excluded trials evaluating behavioural interventions and natural products, which are not regulated by the U.S. Food and Drug Administration (FDA). We evaluated trials on 3 criteria of informativeness: potential redundancy (comparing trial phase, type, patient-participant characteristics, treatment regimen, comparator arms and primary outcome), trials design (according to the recommendations set-out in the May 2020 FDA guidance document on SARS-CoV-2 treatment and prevention trials) and feasibility of patient-participant recruitment (based on timeliness and success of recruitment).
Results
We included all 500 eligible trials in our cohort, 58% of which were Phase 2 and 84.8% were directed towards the treatment of SARS-CoV-2. Close to one third of trials met all three criteria and were deemed informative (29.9% (95% Confidence Interval 23.7–36.9)). The proportion of potentially redundant trials in our cohort was 4.1%. Over half of the trials in our cohort (56.2%) did not meet our criteria for high quality trial design. The proportion of trials with infeasible patient-participant recruitment was 22.6%.
Conclusions
Less than one third of COVID-19 trials registered on ClinicalTrials.gov during the first six months met all three criteria for informativeness. Shortcomings in trial design, recruitment feasibility and redundancy reflect longstanding weaknesses in the clinical research enterprise that were likely amplified by the exceptional circumstances of a pandemic.
Article activity feed
-
-
-
SciScore for 10.1101/2021.08.25.21262155: (What is this?)
Please note, not all rigor criteria are appropriate for all manuscripts.
Table 1: Rigor
Ethics IRB: Our study was not subject to Institutional Review Board approval, as it relies on publicly accessible data and did not involve interaction with research participants. Sex as a biological variable not detected. Randomization Based on the U.S. Food & Drug Association (FDA) May 2020 guidance document for SARS-CoV-2 drug and biological treatment and prevention trials,14 we considered a trial to be well-designed if it was randomized, placebo-controlled (with appropriate standard of care in all arms), double-blinded and included participants aged 60 years or over (as a proxy for an at-risk population). Blinding not detected. Power Analysis We did not perform a sample size calculation, as we … SciScore for 10.1101/2021.08.25.21262155: (What is this?)
Please note, not all rigor criteria are appropriate for all manuscripts.
Table 1: Rigor
Ethics IRB: Our study was not subject to Institutional Review Board approval, as it relies on publicly accessible data and did not involve interaction with research participants. Sex as a biological variable not detected. Randomization Based on the U.S. Food & Drug Association (FDA) May 2020 guidance document for SARS-CoV-2 drug and biological treatment and prevention trials,14 we considered a trial to be well-designed if it was randomized, placebo-controlled (with appropriate standard of care in all arms), double-blinded and included participants aged 60 years or over (as a proxy for an at-risk population). Blinding not detected. Power Analysis We did not perform a sample size calculation, as we included all trials meeting our eligibility criteria within our designated sampling timeframe. Table 2: Resources
Software and Algorithms Sentences Resources Tools and data synthesis: We performed data extraction using Numbat Systematic Review Manager v. 2.11 (RRID:SCR_019207). Numbatdetected: Numbat ( RRID:SCR_019207)16 All analyses were performed using R version 3.6.3.17 We retrieved historical versions of ClinicalTrials.gov using Clinical Trials History Scraper (RRID:SCR_019229). Clinical Trials History Scraperdetected: Clinical Trials History Scraper ( RRID:SCR_019229)Results from OddPub: Thank you for sharing your data.
Results from LimitationRecognizer: An explicit section about the limitations of the techniques employed in this study was not found. We encourage authors to address study limitations.Results from TrialIdentifier: No clinical trial numbers were referenced.
Results from Barzooka: We did not find any issues relating to the usage of bar graphs.
Results from JetFighter: We did not find any issues relating to colormaps.
Results from rtransparent:- Thank you for including a conflict of interest statement. Authors are encouraged to include this statement when submitting to a journal.
- Thank you for including a funding statement. Authors are encouraged to include this statement when submitting to a journal.
- No protocol registration statement was detected.
Results from scite Reference Check: We found no unreliable references.
-