Tocilizumab and Mortality in Hospitalised Patients with Covid-19. A Systematic Review Comparing Randomised Trials with Observational Studies
This article has been Reviewed by the following groups
Listed in
- Evaluated articles (ScreenIT)
Abstract
Background: Early observational studies suggested that tocilizumab might produce clinical improvement in covid-19 patients leading to the use of tocilizumab. Early underpowered randomised controlled trials (RCTs) however did not show benefit until the most recent largest trial. RECOVERY trial. We aimed to compare the evidence from RCTs and observational studies of the effect of tocilizumab on in-hospital mortality in patients with covid-19. Materials and Methods: Embase and PubMed were searched from July 2020 until 1 March 2021. Observational studies and RCTs assessing in-hospital mortality in patients receiving tocilizumab compared with standard care or placebo were included. The primary outcome was in-hospital mortality closest to 30 days. The risk of bias in observational studies was assessed using the ROBINS-I tool. A fixed effect meta-analysis was used to combine relative risks, with random effects and risk of bias as a sensitivity analysis. Results: Of 5,792 publications screened for inclusion, eight RCTs and 33 observational studies were identified. The RCTs showed an overall relative risk reduction in in-hospital mortality at 30 days of 0.86 (95% confidence interval (CI) 0.78 to 0.96) with no statistically significant heterogeneity. 23 of the observational studies had a severe risk of bias, 10 of which did not adjust for potential confounders. The 10 observational studies with moderate risk of bias reported a larger reduction in mortality at 30-days (relative risk 0.72, 95% CI 0.64 to 0.81) but with significant heterogeneity (P<0.01). Conclusion: This meta-analysis provides strong evidence from RCTs that tocilizumab reduces the risk of mortality in hospitalised covid-19 patients. Observational studies with moderate risk of bias exaggerated the benefits on mortality two-fold and showed heterogeneity. Collectively observational studies provide a less reliable evidence base for evaluating treatments for covid-19.
Article activity feed
-
-
SciScore for 10.1101/2021.04.23.21255815: (What is this?)
Please note, not all rigor criteria are appropriate for all manuscripts.
Table 1: Rigor
Ethics not detected. Sex as a biological variable not detected. Randomization Search terms for treatment comparisons and clinical outcomes were combined with search terms for study design (randomised controlled trials and observational studies separately) (see Supplementary file 1). Blinding not detected. Power Analysis not detected. Table 2: Resources
Software and Algorithms Sentences Resources Search Strategy: A search of PubMed and Embase was conducted monthly from July 2020 until 1 March 2021, written in English, Spanish, French, and German, of treatment comparisons in hospitalised covid-19 patients and clinical outcomes. PubMedsuggested: (PubMed, RRID:SCR_004846)Embasesuggested: (EMBASE, RRID:SCR_001650)Wh… SciScore for 10.1101/2021.04.23.21255815: (What is this?)
Please note, not all rigor criteria are appropriate for all manuscripts.
Table 1: Rigor
Ethics not detected. Sex as a biological variable not detected. Randomization Search terms for treatment comparisons and clinical outcomes were combined with search terms for study design (randomised controlled trials and observational studies separately) (see Supplementary file 1). Blinding not detected. Power Analysis not detected. Table 2: Resources
Software and Algorithms Sentences Resources Search Strategy: A search of PubMed and Embase was conducted monthly from July 2020 until 1 March 2021, written in English, Spanish, French, and German, of treatment comparisons in hospitalised covid-19 patients and clinical outcomes. PubMedsuggested: (PubMed, RRID:SCR_004846)Embasesuggested: (EMBASE, RRID:SCR_001650)Where possible MeSH or index terms were used. MeSHsuggested: (MeSH, RRID:SCR_004750)Results from OddPub: We did not detect open data. We also did not detect open code. Researchers are encouraged to share open data when possible (see Nature blog).
Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:Our systematic review has some limitations. We have only evaluated treatment effects on mortality, whereas other outcomes such as time to recovery and need for mechanical ventilation may have an important bearing on the overall benefit profile of tocilizumab and its cost-effectiveness by reducing the duration of illness. We believe that in-hospital mortality, as well as being the most important outcome, provides the least scope for bias in comparing RCTs and observational studies. We have concentrated on overall mortality in all patients, whereas there could be subgroups for whom the survival benefit is more (or less) marked, although subgroup analyses according to severity in the RECOVERY trial suggest that that is not the case. The role of observational studies of treatments in covid-19, and more generally, is controversial. For tocilizumab, the pooled observational studies agree with the RCTs in the direction of benefit on mortality but exaggerated its magnitude two-fold. The large observational studies may seem to have been more informative motivation than the early underpowered RCTs which even when pooled showed no evidence of tocilizumab’s efficacy. We did not combine RCTs and observational studies with network meta-analysis, which may produce highly misleading results (51). The results of observational studies should be used mainly to generate hypotheses and to inform the design of RCTs and not as a basis for treating patients, except when RCTs are not reliable, as rec...
Results from TrialIdentifier: No clinical trial numbers were referenced.
Results from Barzooka: We did not find any issues relating to the usage of bar graphs.
Results from JetFighter: We did not find any issues relating to colormaps.
Results from rtransparent:- Thank you for including a conflict of interest statement. Authors are encouraged to include this statement when submitting to a journal.
- Thank you for including a funding statement. Authors are encouraged to include this statement when submitting to a journal.
- No protocol registration statement was detected.
Results from scite Reference Check: We found no unreliable references.
-
