Credible learning of hydroxychloroquine and dexamethasone effects on COVID-19 mortality outside of randomized trials

  1. Chad Hazlett
  2. David Ami Wulf
  3. Bogdan Pasaniuc
  4. Onyebuchi A. Arah
  5. Kristine M. Erlandson
  6. Brian T. Montague

Reviewed by

Read the full article See related articles

Listed in

Log in to save this article

Abstract

Objectives

To investigate the effectiveness of hydroxychloroquine and dexamethasone on coronavirus disease (COVID-19) mortality using patient data outside of randomized trials.

Design

Phenotypes derived from electronic health records were analyzed using the stability-controlled quasi-experiment (SCQE) to provide a range of possible causal effects of hydroxy-chloroquine and dexamethasone on COVID-19 mortality.

Setting and participants

Data from 2,007 COVID-19 positive patients hospitalized at a large university hospital system over the course of 200 days and not enrolled in randomized trials were analyzed using SCQE. For hyrdoxychloroquine, we examine a high-use cohort (n=766, days 1 to 43) and a later, low-use cohort (n=548, days 44 to 82). For dexamethasone, we examine a low-use cohort (n=614, days 44 to 101) and high-use cohort (n=622, days 102 to 200).

Outcome measure

14-day mortality, with a secondary outcome of 28-day mortality.

Results

Hydroxycholoroquine could only have been significantly (p<0.05) beneficial if baseline mortality was at least 6.4 percentage points (55%) lower among patients in the later (low-use) than the earlier (high-use) cohort. Hydroxychloroquine instead proves significantly harmful if baseline mortality rose from one cohort to the next by just 0.3 percentage points. Dexamethasone significantly reduced mortality risk if baseline mortality in the later (high-use) cohort (days 102-200) was higher than, the same as, or up to 1.5 percentage points lower than that in the earlier (low-use) cohort (days 44-101). It could only prove significantly harmful if mortality improved from one cohort to the next by 6.8 percentage points due to other causes—an assumption implying an unlikely 84% reduction in mortality due to other causes, leaving an in-hospital mortality rate of just 1.3%.

Conclusions

The assumptions required for a beneficial effect of hydroxychloroquine on 14 day mortality are difficult to sustain, while the assumptions required for hydroxychloroquine to be harmful are difficult to reject with confidence. Dexamethasone, by contrast, was beneficial under a wide range of plausible assumptions, and was only harmful if a nearly impossible assumption is met. More broadly, the SCQE reveals what inferences can be credibly supported by evidence from non-randomized uses of experimental therapies, making it a useful tool when randomized trials have not yet produced clear evidence or to provide corroborative evidence from different populations.

Article activity feed

  1. ScreenIT

    SciScore for 10.1101/2020.12.06.20244798: (What is this?)

    Please note, not all rigor criteria are appropriate for all manuscripts.

    Table 1: Rigor

    Institutional Review Board Statementnot detected.
    Randomizationnot detected.
    Blindingnot detected.
    Power Analysisnot detected.
    Sex as a biological variablenot detected.

    Table 2: Resources

    No key resources detected.

    Results from OddPub: We did not detect open data. We also did not detect open code. Researchers are encouraged to share open data when possible (see Nature blog).

    Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:
    4.1 Limitations: The central limitation of this study and approach is also its strength: it avoids providing a narrow claim, because it avoids relying on a narrow assumption that is unlikely to be defensible. This may remain unsatisfying for readers accustomed to more specific claims. However, the types of estimates offered by conventional approaches obtain their apparent specificity not by providing more information, but by concealing how our estimate depend upon on the value of uncertain assumptions, thus risking over-confidence in unsupported conclusions. The SCQE approach serves to communicate what can (not) be claimed subject to what assumption, leaving the reader to argue positively for the assumptions that would be required to reach a conclusion and illustrating the limits of our knowledge. Another limitation, specific to this study, regards sample size. While the sample is larger than those in some randomized trials, and more than sufficient for SCQE mechanically, the estimated effect has to be relatively large (roughly 10 percentage points or more) for the 95% confidence interval to exclude zero. This in turn means that our conclusions will be less decisive over a given plausible range of baseline trends than they may have been with similar estimates but a larger sample. Finally, in both of the studies, the cohorts we defined were largely similar in their composition and use of other treatments, which is not necessary but makes it far easier to reason about plausible...

    Results from TrialIdentifier: No clinical trial numbers were referenced.

    Results from Barzooka: We did not find any issues relating to the usage of bar graphs.

    Results from JetFighter: We did not find any issues relating to colormaps.

    Results from rtransparent:
    • Thank you for including a conflict of interest statement. Authors are encouraged to include this statement when submitting to a journal.
    • Thank you for including a funding statement. Authors are encouraged to include this statement when submitting to a journal.
    • No protocol registration statement was detected.

    About SciScore

    SciScore is an automated tool that is designed to assist expert reviewers by finding and presenting formulaic information scattered throughout a paper in a standard, easy to digest format. SciScore checks for the presence and correctness of RRIDs (research resource identifiers), and for rigor criteria such as sex and investigator blinding. For details on the theoretical underpinning of rigor criteria and the tools shown here, including references cited, please follow this link.

  2. Version published to 10.1101/2020.12.06.20244798v1 on medRxiv