High-cited favorable studies for COVID-19 treatments ineffective in large trials

  1. John P.A. Ioannidis

Reviewed by

Read the full article See related articles

Discuss this preprint

Start a discussion What are Sciety discussions?

Listed in

Log in to save this article

Abstract

No abstract available

Article activity feed

  1. Version published to 10.1016/j.jclinepi.2022.04.001
  2. ScreenIT

    SciScore for 10.1101/2022.01.11.22269097: (What is this?)

    Please note, not all rigor criteria are appropriate for all manuscripts.

    Table 1: Rigor

    Ethicsnot detected.
    Sex as a biological variablenot detected.
    RandomizationThe relative risks for the mortality outcome in RECOVERY was 1.03 (95% confidence interval [CI] 0.91-1.17) based on 5040 randomized participants for lopinavir-ritonavir,9 1.09 (95% CI, 0.97-1.23) based on 4716 randomized participants for hydroxychloroquine,10 0.97 (95% CI, 0.87-1.07) based on 7763 randomized participants for azithromycin,11 1.00 (95% CI, 0.93-1.07) based on 11558 randomized participants for convalescent plasma,12 and 1.01 (95% CI 0.93-1.10) based on 11340 randomized participants for colchicine.
    Blindingnot detected.
    Power Analysisnot detected.

    Table 2: Resources

    No key resources detected.

    Results from OddPub: Thank you for sharing your data.

    Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:
    Some caveats need to be acknowledged. The most important limitation of this analysis is that the large trials may not necessarily be a perfect gold standard. No single clinical study can claim to possess the perfect truth, no matter how well it is conducted and how well it is protected from bias. The CIs of the large trials cannot exclude very small benefits on survival – or small harms. These trials have also shown no benefit also on other outcomes. However, small benefits (or harms) for these outcomes are also not possible to exclude with perfect certainty. Moreover, beneficial effects for some treatments may still exist in circumscribed, special circumstances, with different dosing regimens, and in specific patient subgroups that may have been outside the eligibility criteria of the large RCTs or may have been under-represented in these large RCTs. However, similar concerns and speculative counter-arguments may be raised almost in any clinical topic, especially by those who still believe that a treatment may have merits despite its poor performance in very large trials.61 It should also be acknowledged that not all guidelines have removed these treatments from the list of interventions that they recommend. Remdesivir is probably the most notable example in this regard. It is not recommended by the European Respiratory Society62 and the World Health Organization has issued a conditional recommendation against its use.63 Conversely, the US National Institutes of Health (NIH)...

    Results from TrialIdentifier: No clinical trial numbers were referenced.

    Results from Barzooka: We did not find any issues relating to the usage of bar graphs.

    Results from JetFighter: We did not find any issues relating to colormaps.

    Results from rtransparent:
    • Thank you for including a conflict of interest statement. Authors are encouraged to include this statement when submitting to a journal.
    • Thank you for including a funding statement. Authors are encouraged to include this statement when submitting to a journal.
    • No protocol registration statement was detected.

    Results from scite Reference Check: We found no unreliable references.

    About SciScore

    SciScore is an automated tool that is designed to assist expert reviewers by finding and presenting formulaic information scattered throughout a paper in a standard, easy to digest format. SciScore checks for the presence and correctness of RRIDs (research resource identifiers), and for rigor criteria such as sex and investigator blinding. For details on the theoretical underpinning of rigor criteria and the tools shown here, including references cited, please follow this link.

  3. Version published to 10.1101/2022.01.11.22269097 on medRxiv