Derivation and external validation of a simple risk score to predict in-hospital mortality in patients hospitalized for COVID-19

  1. Charlotte Z. Mann
  2. Chelsea Abshire
  3. Monica Yost
  4. Scott Kaatz
  5. Lakshmi Swaminathan
  6. Scott A. Flanders
  7. Hallie C. Prescott
  8. Johann A. Gagnon-Bartsch

Reviewed by

Read the full article

Listed in

Log in to save this article

Abstract

As severe acute respiratory syndrome coronavirus 2 continues to spread, easy-to-use risk models that predict hospital mortality can assist in clinical decision making and triage. We aimed to develop a risk score model for in-hospital mortality in patients hospitalized with 2019 novel coronavirus (COVID-19) that was robust across hospitals and used clinical factors that are readily available and measured standardly across hospitals.

In this retrospective observational study, we developed a risk score model using data collected by trained abstractors for patients in 20 diverse hospitals across the state of Michigan (Mi-COVID19) who were discharged between March 5, 2020 and August 14, 2020. Patients who tested positive for severe acute respiratory syndrome coronavirus 2 during hospitalization or were discharged with an ICD-10 code for COVID-19 (U07.1) were included. We employed an iterative forward selection approach to consider the inclusion of 145 potential risk factors available at hospital presentation. Model performance was externally validated with patients from 19 hospitals in the Mi-COVID19 registry not used in model development. We shared the model in an easy-to-use online application that allows the user to predict in-hospital mortality risk for a patient if they have any subset of the variables in the final model.

Two thousand one hundred and ninety-three patients in the Mi-COVID19 registry met our inclusion criteria. The derivation and validation sets ultimately included 1690 and 398 patients, respectively, with mortality rates of 19.6% and 18.6%, respectively. The average age of participants in the study after exclusions was 64 years old, and the participants were 48% female, 49% Black, and 87% non-Hispanic. Our final model includes the patient's age, first recorded respiratory rate, first recorded pulse oximetry, highest creatinine level on day of presentation, and hospital's COVID-19 mortality rate. No other factors showed sufficient incremental model improvement to warrant inclusion. The area under the receiver operating characteristics curve for the derivation and validation sets were .796 (95% confidence interval, .767–.826) and .829 (95% confidence interval, .782–.876) respectively.

We conclude that the risk of in-hospital mortality in COVID-19 patients can be reliably estimated using a few factors, which are standardly measured and available to physicians very early in a hospital encounter.

Article activity feed

  1. Version published to 10.1097/md.0000000000027422
  2. ScreenIT

    SciScore for 10.1101/2021.05.04.21256599: (What is this?)

    Please note, not all rigor criteria are appropriate for all manuscripts.

    Table 1: Rigor

    NIH rigor criteria are not applicable to paper type.

    Table 2: Resources

    No key resources detected.

    Results from OddPub: We did not detect open data. We also did not detect open code. Researchers are encouraged to share open data when possible (see Nature blog).

    Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:
    Our study should be interpreted in the context of some limitations. Many of the hospitalizations in our dataset were in Southeastern Michigan during the spring 2020 COVID-19 surge, when many hospitals in this region were experiencing very high patient volumes and treatment differed from current best practices. For example, in March and April 2020, dexamethasone and remdesivir were used only rarely, while hydroxychloroquine use was common. Thus, because our model was developed and validated using data from the Spring 2020 surge, it may overestimate the in-hospital mortality of patients treated in non-surge settings and with current best practices. Importantly, however, our model includes the hospital’s mortality rate for COVID-19 as a predictor, such that the model automatically re-calibrates over time. Furthermore, while in-hospital mortality has changed over time in relation to patient volume and the introduction of new therapies, we expect that age, respiratory rate, pulse oximetry, and creatinine will remain important predictors of in-hospital mortality for COVID-19, as these variables are consistently identified for inclusion in risk-prediction models. However, future studies that evaluate the model discrimination and calibration for patients hospitalized after the summer of 2020 will need to confirm that the model performance does not degrade over time. In sum, we developed a parsimonious risk-prediction model for in-hospital mortality in patients from COVID-19. The use ...

    Results from TrialIdentifier: No clinical trial numbers were referenced.

    Results from Barzooka: We did not find any issues relating to the usage of bar graphs.

    Results from JetFighter: We did not find any issues relating to colormaps.

    Results from rtransparent:
    • Thank you for including a conflict of interest statement. Authors are encouraged to include this statement when submitting to a journal.
    • Thank you for including a funding statement. Authors are encouraged to include this statement when submitting to a journal.
    • No protocol registration statement was detected.

    Results from scite Reference Check: We found no unreliable references.

    About SciScore

    SciScore is an automated tool that is designed to assist expert reviewers by finding and presenting formulaic information scattered throughout a paper in a standard, easy to digest format. SciScore checks for the presence and correctness of RRIDs (research resource identifiers), and for rigor criteria such as sex and investigator blinding. For details on the theoretical underpinning of rigor criteria and the tools shown here, including references cited, please follow this link.

  3. Version published to 10.1101/2021.05.04.21256599v1 on medRxiv