A chest radiography-based artificial intelligence deep-learning model to predict severe Covid-19 patient outcomes: the CAPE (Covid-19 AI Predictive Engine) Model
This article has been Reviewed by the following groups
Discuss this preprint
Start a discussion What are Sciety discussions?Listed in
- Evaluated articles (ScreenIT)
Abstract
Background
Chest radiography may be used together with deep-learning models to prognosticate COVID-19 patient outcomes
Purpose
T o evaluate the performance of a deep-learning model for the prediction of severe patient outcomes from COVID-19 pneumonia on chest radiographs.
Methods
A deep-learning model (CAPE: Covid-19 AI Predictive Engine) was trained on 2337 CXR images including 2103 used only for validation while training. The prospective test set consisted of CXR images (n=70) obtained from RT-PCR confirmed COVID-19 pneumonia patients between 1 January and 30 April 2020 in a single center. The radiographs were analyzed by the AI model. Model performance was obtained by receiver operating characteristic curve analysis.
Results
In the prospective test set, the mean age of the patients was 46 (+/-16.2) years (84.2% male). The deep-learning model accurately predicted outcomes of ICU admission/mortality from COVID-19 pneumonia with an AUC of 0.79 (95% CI 0.79-0.96). Compared to traditional risk scoring systems for pneumonia based upon laboratory and clinical parameters, the model matched the EWS and MulBTSA risk scoring systems and outperformed CURB-65.
Conclusions
A deep-learning model was able to predict severe patient outcomes (ICU admission and mortality) from COVID-19 on chest radiographs.
Key Results
A deep-learning model was able to predict severe patient outcomes (ICU admission and mortality) from COVID-19 from chest radiographs with an AUC of 0.79, which is comparable to traditional risk scoring systems for pneumonia.
Summary Statement
This is a chest radiography-based AI model to prognosticate the risk of severe COVID-19 pneumonia outcomes.
Article activity feed
-
SciScore for 10.1101/2020.05.25.20113084: (What is this?)
Please note, not all rigor criteria are appropriate for all manuscripts.
Table 1: Rigor
Institutional Review Board Statement not detected. Randomization For the training and validation set, we randomly selected 90% [n=2103] of inpatient episodes for model training and held out 10% for validation. Blinding Model development - Training, Validation and Testing: A retrospective study sample of chest radiographs from adult patients admitted to [blinded for submission] hospital from 1 January 2019 to 31 December 2019 was used for the development of the predictive model (Figure 1). Power Analysis not detected. Sex as a biological variable not detected. Table 2: Resources
Software and Algorithms Sentences Resources The model was implemented in Keras, version 1.3.0 29 and Scikit-learn, version 0.19.1 and … SciScore for 10.1101/2020.05.25.20113084: (What is this?)
Please note, not all rigor criteria are appropriate for all manuscripts.
Table 1: Rigor
Institutional Review Board Statement not detected. Randomization For the training and validation set, we randomly selected 90% [n=2103] of inpatient episodes for model training and held out 10% for validation. Blinding Model development - Training, Validation and Testing: A retrospective study sample of chest radiographs from adult patients admitted to [blinded for submission] hospital from 1 January 2019 to 31 December 2019 was used for the development of the predictive model (Figure 1). Power Analysis not detected. Sex as a biological variable not detected. Table 2: Resources
Software and Algorithms Sentences Resources The model was implemented in Keras, version 1.3.0 29 and Scikit-learn, version 0.19.1 and Python, version 3.7 (Python Software Foundation). Scikit-learnsuggested: (scikit-learn, RRID:SCR_002577)Pythonsuggested: (IPython, RRID:SCR_001658)Results from OddPub: We did not detect open data. We also did not detect open code. Researchers are encouraged to share open data when possible (see Nature blog).
Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:We identified several limitations in our study. Firstly, the training and validation sets were obtained from a single institution, which may not be representative of other institutions. Secondly, the number of RT-PCR confirmed COVID-19 chest radiographs in the test set was relatively small (n=70), relative to the number of non-COVID-19 pneumonia images (n=2736). To improve the performance of the deep-learning predictive model for COVID-19, a large training dataset of RT-PCR confirmed COVID-19 chest radiographs is needed. The model could also be improved by adopting a multimodal approach by combining radiographic imaging data with clinical and laboratory results for model training. In conclusion, we evaluated a deep-learning AI model (CAPE: COVID-19 AI Predictive Engine) for the prediction of COVID-19 severe outcomes, namely ICU admission and mortality. The CAPE AI model’s performance was comparable and at times superior to traditional prognostic risk scoring systems for pneumonia. This tool will be made available in the future free-of-charge on the author’s website upon request, to aid in the global health response to COVID-19.
Results from TrialIdentifier: No clinical trial numbers were referenced.
Results from Barzooka: We did not find any issues relating to the usage of bar graphs.
Results from JetFighter: We did not find any issues relating to colormaps.
Results from rtransparent:- Thank you for including a conflict of interest statement. Authors are encouraged to include this statement when submitting to a journal.
- Thank you for including a funding statement. Authors are encouraged to include this statement when submitting to a journal.
- No protocol registration statement was detected.
-
