A predictive model to estimate survival of hospitalized COVID-19 patients from admission data

  1. Todd J. Levy
  2. Safiya Richardson
  3. Kevin Coppa
  4. Douglas P. Barnaby
  5. Thomas McGinn
  6. Lance B. Becker
  7. Karina W. Davidson
  8. Jamie S. Hirsch
  9. Theodoros P. Zanos
  10. And the Northwell COVID-19 Research Consortium
  11. Stuart L. Cohen
  12. Shubham Debnath
  13. Andrew J. Dominello
  14. Louise Falzon
  15. Michael Gitman
  16. Eun-Ji Kim
  17. Alexander Makhnevich
  18. Jazmin N. Mogavero
  19. Ernesto P. Molmenti
  20. Marc d. Paradis
  21. Viktor Tóth

Reviewed by

Read the full article

Listed in

Log in to save this article

Abstract

Objective

Our primary objective was to use initial data available to clinicians to characterize and predict survival for hospitalized coronavirus disease 2019 (COVID-19) patients. While clinical characteristics and mortality risk factors of COVID-19 patients have been reported, a practical survival calculator based on data from a diverse group of U.S. patients has not yet been introduced. Such a tool would provide timely and valuable guidance in decision-making during this global pandemic.

Design

We extracted demographic, laboratory, clinical, and treatment data from electronic health records and used it to build and test the predictive accuracy of a survival probability calculator referred to as “the Northwell COVID-19 Survival (‘NOCOS’) calculator.”

Setting

13 acute care facilities at Northwell Health served as the setting for this study.

Participants

5,233 hospitalized COVID-19–positive patients served as the participants for this study.

Main outcome measures

The NOCOS calculator was constructed using multivariate regression with L1 regularization (LASSO) to predict survival during hospitalization. Model predictive performance was measured using Receiver Operating Characteristic (ROC) curves and the Area Under the Curve (AUC) of the calculators.

Results

Patient age, serum blood urea nitrogen, Emergency Severity Index, red cell distribution width, absolute neutrophil count, serum bicarbonate, and glucose were identified as the optimal predictors of survival by multivariate LASSO regression. The predictive performance of the NOCOS calculator had an AUC of 0.832, reaching 0.91 when updated for each patient daily, with stability assessed and maintained for 14 consecutive days. This outperformed other established models, including the Sequential Organ Failure Assessment (SOFA) score (0.732).

Conclusions

We present a practical estimate of survival probability that outperforms other general risk models. The seven early predictors of in-hospital survival can help clinicians identify patients with increased probabilities of survival and provide critical decision support as COVID-19 spreads across the U.S.

Trial registration

N/A

Article activity feed

  1. ScreenIT

    SciScore for 10.1101/2020.04.22.20075416: (What is this?)

    Please note, not all rigor criteria are appropriate for all manuscripts.

    Table 1: Rigor

    Institutional Review Board StatementIRB: This study was approved by the Institutional Review Boards at Northwell Health and Maimonides Medical Center as minimal-risk research that used data collected for routine clinical practice, and as such, waived the requirement for informed consent.
    Consent: This study was approved by the Institutional Review Boards at Northwell Health and Maimonides Medical Center as minimal-risk research that used data collected for routine clinical practice, and as such, waived the requirement for informed consent.
    Randomizationnot detected.
    Blindingnot detected.
    Power Analysisnot detected.
    Sex as a biological variablenot detected.

    Table 2: Resources

    Software and Algorithms
    SentencesResources
    All analyses were performed in MATLAB 2019b (The Mathworks, Inc., Natick, MA).
    MATLAB
    suggested: (MATLAB, RRID:SCR_001622)

    Results from OddPub: We did not detect open data. We also did not detect open code. Researchers are encouraged to share open data when possible (see Nature blog).

    Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:
    Limitations: The study population only included patients within the New York City metropolitan area. However, given the diverse demographic population of the region, we expect the model to generalize to patients at centers outside of this geographic area. The data were collected entirely from EHR reports, which supported robust and rapid analysis of a large cohort of patients. However, we did not include data elements that would require manual chart review. Due to the retrospective study design, not all laboratory tests were completed on all patients, and the performance of these variables could not be adequately assessed. To optimize for usability and portability, the analysis was designed to be linear and to include a minimum number of predictors. Non-linear or convolutional/recurrent models may provide improved performance but might not be easily used at all centers. Conclusion: This study is the first to develop and externally validate a simple predictive model of survival for hospitalized patients with COVID-19 based on structured, objective data that is routinely available at admission in the United States. Serum blood urea nitrogen, age, absolute neutrophil count, red cell distribution width, oxygen saturation, and serum sodium were identified as the 6 optimal predictors of survival. The NOCOS Calculator can predict survival more accurately than commonly used survival predictors.

    Results from TrialIdentifier: No clinical trial numbers were referenced.

    Results from Barzooka: We did not find any issues relating to the usage of bar graphs.

    Results from JetFighter: We did not find any issues relating to colormaps.

    Results from rtransparent:
    • Thank you for including a conflict of interest statement. Authors are encouraged to include this statement when submitting to a journal.
    • Thank you for including a funding statement. Authors are encouraged to include this statement when submitting to a journal.
    • No protocol registration statement was detected.

    About SciScore

    SciScore is an automated tool that is designed to assist expert reviewers by finding and presenting formulaic information scattered throughout a paper in a standard, easy to digest format. SciScore checks for the presence and correctness of RRIDs (research resource identifiers), and for rigor criteria such as sex and investigator blinding. For details on the theoretical underpinning of rigor criteria and the tools shown here, including references cited, please follow this link.

  2. Version published to 10.1101/2020.04.22.20075416v3 on medRxiv
  3. Version published to 10.1101/2020.04.22.20075416v2 on medRxiv
  4. Version published to 10.1101/2020.04.22.20075416v1 on medRxiv