Development of a clinical decision support system for severity risk prediction and triage of COVID-19 patients at hospital admission: an international multicentre study

  1. Guangyao Wu
  2. Pei Yang
  3. Yuanliang Xie
  4. Henry C. Woodruff
  5. Xiangang Rao
  6. Julien Guiot
  7. Anne-Noelle Frix
  8. Renaud Louis
  9. Michel Moutschen
  10. Jiawei Li
  11. Jing Li
  12. Chenggong Yan
  13. Dan Du
  14. Shengchao Zhao
  15. Yi Ding
  16. Bin Liu
  17. Wenwu Sun
  18. Fabrizio Albarello
  19. Alessandra D'Abramo
  20. Vincenzo Schininà
  21. Emanuele Nicastri
  22. Mariaelena Occhipinti
  23. Giovanni Barisione
  24. Emanuela Barisione
  25. Iva Halilaj
  26. Pierre Lovinfosse
  27. Xiang Wang
  28. Jianlin Wu
  29. Philippe Lambin

Reviewed by

Read the full article See related articles

Listed in

Log in to save this article

Abstract

The outbreak of coronavirus disease 2019 (COVID-19) has globally strained medical resources and caused significant mortality.

Objective

To develop and validate a machine-learning model based on clinical features for severity risk assessment and triage for COVID-19 patients at hospital admission.

Method

725 patients were used to train and validate the model. This included a retrospective cohort from Wuhan, China of 299 hospitalised COVID-19 patients from 23 December 2019 to 13 February 2020, and five cohorts with 426 patients from eight centres in China, Italy and Belgium from 20 February 2020 to 21 March 2020. The main outcome was the onset of severe or critical illness during hospitalisation. Model performances were quantified using the area under the receiver operating characteristic curve (AUC) and metrics derived from the confusion matrix.

Results

In the retrospective cohort, the median age was 50 years and 137 (45.8%) were male. In the five test cohorts, the median age was 62 years and 236 (55.4%) were male. The model was prospectively validated on five cohorts yielding AUCs ranging from 0.84 to 0.93, with accuracies ranging from 74.4% to 87.5%, sensitivities ranging from 75.0% to 96.9%, and specificities ranging from 55.0% to 88.0%, most of which performed better than the pneumonia severity index. The cut-off values of the low-, medium- and high-risk probabilities were 0.21 and 0.80. The online calculators can be found at www.covid19risk.ai .

Conclusion

The machine-learning model, nomogram and online calculator might be useful to access the onset of severe and critical illness among COVID-19 patients and triage at hospital admission.

Article activity feed

  1. Version published to 10.1183/13993003.01104-2020
  2. ScreenIT

    SciScore for 10.1101/2020.05.01.20053413: (What is this?)

    Please note, not all rigor criteria are appropriate for all manuscripts.

    Table 1: Rigor

    Institutional Review Board StatementIRB: Patients: The institutional review boards of The Central Hospital of Wuhan (No.2020-71) approved this study, which followed the Standards for Reporting of Diagnostic Accuracy Studies statement,10 and the requirement for written informed consent was waived.
    Consent: Patients: The institutional review boards of The Central Hospital of Wuhan (No.2020-71) approved this study, which followed the Standards for Reporting of Diagnostic Accuracy Studies statement,10 and the requirement for written informed consent was waived.
    RandomizationIn order to gauge the level of overfitting, the outcomes were randomized on the best model.
    Blinding13 The semantic CT characteristics (including ground-glass opacity, consolidation, vascular enlargement, air bronchogram, and lesion range score) were independently evaluated on all datasets by two radiologists (PY [a radiologist with 5 years’ experience in chest CT images] and YX [a radiologist with 20 years’ experience in chest CT images]), who were blinded to clinical and laboratory results.
    Power Analysisnot detected.
    Sex as a biological variablenot detected.

    Table 2: Resources

    No key resources detected.

    Results from OddPub: We did not detect open data. We also did not detect open code. Researchers are encouraged to share open data when possible (see Nature blog).

    Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:
    Limitations: Our study has several limitations. First, selection bias is unavoidable due to its retrospective modeling and the limited and unbalanced sample size. Second, patients from different races and ethnicities may have diverse clinical and laboratory results, and the self-medication of patients before admission may affect the clinical and laboratory results. Third, the threshold to go to the hospital can vary from country to country, we are also aware that RNA viruses can mutate rapidly and that could have an impact of the performance of the models. We therefore propose that those models should be continuously updated for example using privacy-preserving distributed learning approaches.29,30 Fourth, the CT features used for this study are semantic features from the first CT scan, and quantitative features automatically extracted from CT images using radiomics or deep learning approaches may improve its prognostic performance, and follow-up CT scan may yield more information. Finally, there is also the fundamental weakness of nomograms, which do not give a confidence interval to the final output. Conclusion: Elderly COVID-19 patients with hypertension and non-hospital staff seem more vulnerable to develop a severe illness as per defining criteria, which can cause a wide range of laboratory and CT anomalies. Furthermore, our model based on lactate dehydrogenase, C-reactive protein, calcium, age, lymphocyte proportion, urea, and creatine kinase might be a useful prelimina...

    Results from TrialIdentifier: No clinical trial numbers were referenced.

    Results from Barzooka: We did not find any issues relating to the usage of bar graphs.

    Results from JetFighter: We did not find any issues relating to colormaps.

    Results from rtransparent:
    • Thank you for including a conflict of interest statement. Authors are encouraged to include this statement when submitting to a journal.
    • Thank you for including a funding statement. Authors are encouraged to include this statement when submitting to a journal.
    • No protocol registration statement was detected.

    About SciScore

    SciScore is an automated tool that is designed to assist expert reviewers by finding and presenting formulaic information scattered throughout a paper in a standard, easy to digest format. SciScore checks for the presence and correctness of RRIDs (research resource identifiers), and for rigor criteria such as sex and investigator blinding. For details on the theoretical underpinning of rigor criteria and the tools shown here, including references cited, please follow this link.

  3. Version published to 10.1101/2020.05.01.20053413v1 on medRxiv