Machine learning in predicting respiratory failure in patients with COVID-19 pneumonia—Challenges, strengths, and opportunities in a global health emergency

  1. Davide Ferrari
  2. Jovana Milic
  3. Roberto Tonelli
  4. Francesco Ghinelli
  5. Marianna Meschiari
  6. Sara Volpi
  7. Matteo Faltoni
  8. Giacomo Franceschi
  9. Vittorio Iadisernia
  10. Dina Yaacoub
  11. Giacomo Ciusa
  12. Erica Bacca
  13. Carlotta Rogati
  14. Marco Tutone
  15. Giulia Burastero
  16. Alessandro Raimondi
  17. Marianna Menozzi
  18. Erica Franceschini
  19. Gianluca Cuomo
  20. Luca Corradi
  21. Gabriella Orlando
  22. Antonella Santoro
  23. Margherita Digaetano
  24. Cinzia Puzzolante
  25. Federica Carli
  26. Vanni Borghi
  27. Andrea Bedini
  28. Riccardo Fantini
  29. Luca Tabbì
  30. Ivana Castaniere
  31. Stefano Busani
  32. Enrico Clini
  33. Massimo Girardis
  34. Mario Sarti
  35. Andrea Cossarizza
  36. Cristina Mussini
  37. Federica Mandreoli
  38. Paolo Missier
  39. Giovanni Guaraldi

Reviewed by

Read the full article

Listed in

Log in to save this article

Abstract

The aim of this study was to estimate a 48 hour prediction of moderate to severe respiratory failure, requiring mechanical ventilation, in hospitalized patients with COVID-19 pneumonia.

Methods

This was an observational prospective study that comprised consecutive patients with COVID-19 pneumonia admitted to hospital from 21 February to 6 April 2020. The patients’ medical history, demographic, epidemiologic and clinical data were collected in an electronic patient chart. The dataset was used to train predictive models using an established machine learning framework leveraging a hybrid approach where clinical expertise is applied alongside a data-driven analysis. The study outcome was the onset of moderate to severe respiratory failure defined as PaO 2 /FiO 2 ratio <150 mmHg in at least one of two consecutive arterial blood gas analyses in the following 48 hours. Shapley Additive exPlanations values were used to quantify the positive or negative impact of each variable included in each model on the predicted outcome.

Results

A total of 198 patients contributed to generate 1068 usable observations which allowed to build 3 predictive models based respectively on 31-variables signs and symptoms, 39-variables laboratory biomarkers and 91-variables as a composition of the two. A fourth “boosted mixed model” included 20 variables was selected from the model 3, achieved the best predictive performance (AUC = 0.84) without worsening the FN rate. Its clinical performance was applied in a narrative case report as an example.

Conclusion

This study developed a machine model with 84% prediction accuracy, which is able to assist clinicians in decision making process and contribute to develop new analytics to improve care at high technology readiness levels.

Article activity feed

  1. Version published to 10.1371/journal.pone.0239172
  2. ScreenIT

    SciScore for 10.1101/2020.05.30.20107888: (What is this?)

    Please note, not all rigor criteria are appropriate for all manuscripts.

    Table 1: Rigor

    NIH rigor criteria are not applicable to paper type.

    Table 2: Resources

    No key resources detected.

    Results from OddPub: We did not detect open data. We also did not detect open code. Researchers are encouraged to share open data when possible (see Nature blog).

    Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:
    Our study has several limitations. Firstly, it does not take account of radiological findings on X-rays or CT, as these data were not collected consistently during the admission period. Secondly, we chose as outcome a cut-off of PaO2/FiO2 which represents one of the most important criteria for invasive mechanical ventilation, but not mechanical ventilation itself. Also, the model is oblivious to a whole panel of interleukins values, as they were not collected on daily basis regardless of their potential involvement in the development of severe respiratory failure. Lastly, the model will need to evolve with the growth of the dataset, providing a more accurate cut-off risk value. In conclusion, this study developed a machine learning algorithm aimed to assist clinicians in dealing with COVID-19 health emergency. It is proving useful in predicting severe respiratory failure requiring mechanical ventilation in the following 48 hours, allowing to anticipate urgent events potentially improving management of critically ill patients.

    Results from TrialIdentifier: No clinical trial numbers were referenced.

    Results from Barzooka: We did not find any issues relating to the usage of bar graphs.

    Results from JetFighter: We did not find any issues relating to colormaps.

    Results from rtransparent:
    • Thank you for including a conflict of interest statement. Authors are encouraged to include this statement when submitting to a journal.
    • Thank you for including a funding statement. Authors are encouraged to include this statement when submitting to a journal.
    • No protocol registration statement was detected.

    About SciScore

    SciScore is an automated tool that is designed to assist expert reviewers by finding and presenting formulaic information scattered throughout a paper in a standard, easy to digest format. SciScore checks for the presence and correctness of RRIDs (research resource identifiers), and for rigor criteria such as sex and investigator blinding. For details on the theoretical underpinning of rigor criteria and the tools shown here, including references cited, please follow this link.

  3. Version published to 10.1101/2020.05.30.20107888v2 on medRxiv
  4. Version published to 10.1101/2020.05.30.20107888v1 on medRxiv