Using machine learning for predicting intensive care unit resource use during the COVID-19 pandemic in Denmark

  1. Stephan Sloth Lorenzen
  2. Mads Nielsen
  3. Espen Jimenez-Solem
  4. Tonny Studsgaard Petersen
  5. Anders Perner
  6. Hans-Christian Thorsen-Meyer
  7. Christian Igel
  8. Martin Sillesen

Reviewed by

Read the full article See related articles

Listed in

Log in to save this article

Abstract

The COVID-19 pandemic has put massive strains on hospitals, and tools to guide hospital planners in resource allocation during the ebbs and flows of the pandemic are urgently needed. We investigate whether machine learning (ML) can be used for predictions of intensive care requirements a fixed number of days into the future. Retrospective design where health Records from 42,526 SARS-CoV-2 positive patients in Denmark was extracted. Random Forest (RF) models were trained to predict risk of ICU admission and use of mechanical ventilation after n days ( n  = 1, 2, …, 15). An extended analysis was provided for n  = 5 and n  = 10. Models predicted n -day risk of ICU admission with an area under the receiver operator characteristic curve (ROC-AUC) between 0.981 and 0.995, and n -day risk of use of ventilation with an ROC-AUC between 0.982 and 0.997. The corresponding n -day forecasting models predicted the needed ICU capacity with a coefficient of determination (R 2 ) between 0.334 and 0.989 and use of ventilation with an R 2 between 0.446 and 0.973. The forecasting models performed worst, when forecasting many days into the future (for large n ). For n  = 5, ICU capacity was predicted with ROC-AUC 0.990 and R 2 0.928, and use of ventilator was predicted with ROC-AUC 0.994 and R 2 0.854. Random Forest-based modelling can be used for accurate n -day forecasting predictions of ICU resource requirements, when n is not too large.

Article activity feed

  1. Version published to 10.1038/s41598-021-98617-1
  2. ScreenIT

    SciScore for 10.1101/2021.03.19.21253947: (What is this?)

    Please note, not all rigor criteria are appropriate for all manuscripts.

    Table 1: Rigor

    NIH rigor criteria are not applicable to paper type.

    Table 2: Resources

    No key resources detected.

    Results from OddPub: We did not detect open data. We also did not detect open code. Researchers are encouraged to share open data when possible (see Nature blog).

    Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:
    The caveat is, however, that such systems require integration of healthcare and population specific data, enabling extraction of EHR data on a community wide rather than a hospital specific scale. Healthcare systems operating as individual units rather than covering the entirety of a regional population, may thus not be optimally suited for real-time deployment of these prediction models. This study has several limitations. First, we model data retrospectively and have not demonstrated a prospective value in this study. As such, although accurate on retrospective data predictions, novel features of a potential third COVID-19 wave could affect model performance. Furthermore, we model a selected subset of patient-derived variables based on previous experience7, although other data points could affect the model’s classification ability. Secondly, we have not performed external validation on a separate cohort. This, however, may not be desirable due to several factors: The model should be trained to forecast locally, and transferring to different healthcare systems with different social and geographical factors would require retraining of the model to capture these effects. We have also recently demonstrated that transferring COVID-19 ML prediction models between health care systems internationally results in a reduction of the classification precision, presumably due to inherent differences between healthcare systems even though patient demographics are comparable7. Thirdly, as ...

    Results from TrialIdentifier: No clinical trial numbers were referenced.

    Results from Barzooka: We did not find any issues relating to the usage of bar graphs.

    Results from JetFighter: We did not find any issues relating to colormaps.

    Results from rtransparent:
    • Thank you for including a conflict of interest statement. Authors are encouraged to include this statement when submitting to a journal.
    • Thank you for including a funding statement. Authors are encouraged to include this statement when submitting to a journal.
    • No protocol registration statement was detected.

    About SciScore

    SciScore is an automated tool that is designed to assist expert reviewers by finding and presenting formulaic information scattered throughout a paper in a standard, easy to digest format. SciScore checks for the presence and correctness of RRIDs (research resource identifiers), and for rigor criteria such as sex and investigator blinding. For details on the theoretical underpinning of rigor criteria and the tools shown here, including references cited, please follow this link.

  3. Version published to 10.1101/2021.03.19.21253947v1 on medRxiv