Development and validation of a predictive model for critical illness in adult patients requiring hospitalization for COVID-19

  1. Neha Paranjape
  2. Lauren L. Staples
  3. Christina Y. Stradwick
  4. Herman Gene Ray
  5. Ian J. Saldanha

Reviewed by

Read the full article

Listed in

Log in to save this article

Abstract

Identifying factors that can predict severe disease in patients needing hospitalization for COVID-19 is crucial for early recognition of patients at greatest risk.

Objective

(1) Identify factors predicting intensive care unit (ICU) transfer and (2) develop a simple calculator for clinicians managing patients hospitalized with COVID-19.

Methods

A total of 2,685 patients with laboratory-confirmed COVID-19 admitted to a large metropolitan health system in Georgia, USA between March and July 2020 were included in the study. Seventy-five percent of patients were included in the training dataset (admitted March 1 to July 10). Through multivariable logistic regression, we developed a prediction model (probability score) for ICU transfer. Then, we validated the model by estimating its performance accuracy (area under the curve [AUC]) using data from the remaining 25% of patients (admitted July 11 to July 31).

Results

We included 2,014 and 671 patients in the training and validation datasets, respectively. Diabetes mellitus, coronary artery disease, chronic kidney disease, serum C-reactive protein, and serum lactate dehydrogenase were identified as significant risk factors for ICU transfer, and a prediction model was developed. The AUC was 0.752 for the training dataset and 0.769 for the validation dataset. We developed a free, web-based calculator to facilitate use of the prediction model ( https://icucovid19.shinyapps.io/ICUCOVID19/ ).

Conclusion

Our validated, simple, and accessible prediction model and web-based calculator for ICU transfer may be useful in assisting healthcare providers in identifying hospitalized patients with COVID-19 who are at high risk for clinical deterioration. Triage of such patients for early aggressive treatment can impact clinical outcomes for this potentially deadly disease.

Article activity feed

  1. Version published to 10.1371/journal.pone.0248891
  2. ScreenIT

    SciScore for 10.1101/2021.01.22.21250289: (What is this?)

    Please note, not all rigor criteria are appropriate for all manuscripts.

    Table 1: Rigor

    Institutional Review Board StatementConsent: The study was granted exempt status and the requirement for obtaining informed consent was waived by the Wellstar Health System Institutional Review Board (Approval Number: 1611062-1).
    IRB: The study was granted exempt status and the requirement for obtaining informed consent was waived by the Wellstar Health System Institutional Review Board (Approval Number: 1611062-1).
    Randomizationnot detected.
    Blindingnot detected.
    Power Analysisnot detected.
    Sex as a biological variablenot detected.

    Table 2: Resources

    Software and Algorithms
    SentencesResources
    Variables gathered included: Statistical Analysis: We conducted bivariable and multivariable analyses of the associations between various independent variables and the dependent variable of interest (transfer to ICU service).
    Statistical Analysis
    suggested: (Statistical Analysis System, RRID:SCR_008567)
    We used Python® version 3.7.6 for preprocessing and SAS® server version 9.4 for Chi-square tests and regression analyses.
    Python®
    suggested: (IPython, RRID:SCR_001658)
    SAS®
    suggested: (SASqPCR, RRID:SCR_003056)

    Results from OddPub: We did not detect open data. We also did not detect open code. Researchers are encouraged to share open data when possible (see Nature blog).

    Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:
    Strengths and Limitations: Our study has a number of strengths. First, our predictive model included a limited number of variables that are easily available at hospital presentation, including small tertiary care hospitals. Second, we had a large sample size in both the training as well as validation datasets. Third, the demographics of patients in the current study are highly representative of hospitalized patients in the U.S., which are more diverse and include more vulnerable populations than elsewhere.17 Further, our health system includes nine community hospitals with a total capacity of over 2,500 beds. The model we developed may therefore be generalizable to other similar acute care facilities in the U.S. Fourth, apart from laboratory values, our analysis included presence of co-morbidities such as DM, CAD, and CKD that have been shown to be risk factors for severe disease.11 Fifth, we defined the need for ICU as the transfer to ICU service rather than physical location of the patient. Due to bed shortages during the height of the pandemic, hospitals instituted makeshift locations for ICU-level care. We were thus able to accurately capture true ICU transfers. Lastly, we created an easy, ready to use, web-based calculator, freely accessible to clinicians. Our study also has some limitations. First, we developed and validated the prediction model only for hospitalized patients. Thus, the model may not be applicable to non-hospitalized patients with COVID-19. Second, alth...

    Results from TrialIdentifier: No clinical trial numbers were referenced.

    Results from Barzooka: We did not find any issues relating to the usage of bar graphs.

    Results from JetFighter: We did not find any issues relating to colormaps.

    Results from rtransparent:
    • Thank you for including a conflict of interest statement. Authors are encouraged to include this statement when submitting to a journal.
    • Thank you for including a funding statement. Authors are encouraged to include this statement when submitting to a journal.
    • No protocol registration statement was detected.

    About SciScore

    SciScore is an automated tool that is designed to assist expert reviewers by finding and presenting formulaic information scattered throughout a paper in a standard, easy to digest format. SciScore checks for the presence and correctness of RRIDs (research resource identifiers), and for rigor criteria such as sex and investigator blinding. For details on the theoretical underpinning of rigor criteria and the tools shown here, including references cited, please follow this link.

  3. Version published to 10.1101/2021.01.22.21250289v1 on medRxiv