Tracking and predicting COVID-19 radiological trajectory on chest X-rays using deep learning
This article has been Reviewed by the following groups
Listed in
- Evaluated articles (ScreenIT)
Abstract
Radiological findings on chest X-ray (CXR) have shown to be essential for the proper management of COVID-19 patients as the maximum severity over the course of the disease is closely linked to the outcome. As such, evaluation of future severity from current CXR would be highly desirable. We trained a repurposed deep learning algorithm on the CheXnet open dataset (224,316 chest X-ray images of 65,240 unique patients) to extract features that mapped to radiological labels. We collected CXRs of COVID-19-positive patients from an open-source dataset (COVID-19 image data collection) and from a multi-institutional local ICU dataset. The data was grouped into pairs of sequential CXRs and were categorized into three categories: ‘Worse’, ‘Stable’, or ‘Improved’ on the basis of radiological evolution ascertained from images and reports. Classical machine-learning algorithms were trained on the deep learning extracted features to perform immediate severity evaluation and prediction of future radiological trajectory. Receiver operating characteristic analyses and Mann-Whitney tests were performed. Deep learning predictions between “Worse” and “Improved” outcome categories and for severity stratification were significantly different for three radiological signs and one diagnostic (‘Consolidation’, ‘Lung Lesion’, ‘Pleural effusion’ and ‘Pneumonia’; all P < 0.05). Features from the first CXR of each pair could correctly predict the outcome category between ‘Worse’ and ‘Improved’ cases with a 0.81 (0.74–0.83 95% CI) AUC in the open-access dataset and with a 0.66 (0.67–0.64 95% CI) AUC in the ICU dataset. Features extracted from the CXR could predict disease severity with a 52.3% accuracy in a 4-way classification. Severity evaluation trained on the COVID-19 image data collection had good out-of-distribution generalization when testing on the local dataset, with 81.6% of intubated ICU patients being classified as critically ill, and the predicted severity was correlated with the clinical outcome with a 0.639 AUC. CXR deep learning features show promise for classifying disease severity and trajectory. Once validated in studies incorporating clinical data and with larger sample sizes, this information may be considered to inform triage decisions.
Article activity feed
-
-
SciScore for 10.1101/2020.05.01.20086207: (What is this?)
Please note, not all rigor criteria are appropriate for all manuscripts.
Table 1: Rigor
NIH rigor criteria are not applicable to paper type.Table 2: Resources
Software and Algorithms Sentences Resources We removed the following radiological findings from the training set, given their irrelevance to the purpose of our study: “No Findings”; “Fracture”; “Support Devices”. Findings”suggested: NoneSoftware: Deep learning feature extraction was done in Python using the PyTorch library (version 1.4). Pythonsuggested: (IPython, RRID:SCR_001658)Statistical Analysis: Demographics were expressed as mean (standard deviation) in years, and differences between outcome groups were tested using the SciPy library. SciPysuggested: (SciPy, RRID:SCR_008058)Results from OddPub: Thank you for sharing your code …
SciScore for 10.1101/2020.05.01.20086207: (What is this?)
Please note, not all rigor criteria are appropriate for all manuscripts.
Table 1: Rigor
NIH rigor criteria are not applicable to paper type.Table 2: Resources
Software and Algorithms Sentences Resources We removed the following radiological findings from the training set, given their irrelevance to the purpose of our study: “No Findings”; “Fracture”; “Support Devices”. Findings”suggested: NoneSoftware: Deep learning feature extraction was done in Python using the PyTorch library (version 1.4). Pythonsuggested: (IPython, RRID:SCR_001658)Statistical Analysis: Demographics were expressed as mean (standard deviation) in years, and differences between outcome groups were tested using the SciPy library. SciPysuggested: (SciPy, RRID:SCR_008058)Results from OddPub: Thank you for sharing your code and data.
Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:Study Limitations: This study has some limitations. First, the small size of the test dataset, which inevitably must be augmented to avoid potential bias, most notably case selection, and to confirm generalizability. However, this study represents a proof of concept whose predictive performance can be reassessed as the research community shares additional cases of COVID-19-positive CXRs. Second, the time duration between sequential CXRs was not uniform, which may have diminished the appraisal of the features’ sensitivity to change and the predictive ability of our model. Further, the design of the study is retrospective. However, as the pandemic unfolds, new clinical and radiological data will be continuously incorporated in the test set from the open source repositories, and in future cases from the authors’ institutions, which will truly test generalizability and solve most of these limitations. The authors would be grateful to any reader that would be willing to contribute to this effort. To a degree, this report has demonstrated that deep learning features can track radiological progression in COVID-19 but also predict temporal evolution, adding evidence to the conceptualization that there is directional information in static x-rays allowing this prediction. It should be restated however that the reference standard was categorization of imaging rather than clinical outcomes, such as duration of ICU stay or mortality. Hence, it remains to be determined whether these featur...
Results from TrialIdentifier: No clinical trial numbers were referenced.
Results from Barzooka: We did not find any issues relating to the usage of bar graphs.
Results from JetFighter: We did not find any issues relating to colormaps.
Results from rtransparent:- Thank you for including a conflict of interest statement. Authors are encouraged to include this statement when submitting to a journal.
- Thank you for including a funding statement. Authors are encouraged to include this statement when submitting to a journal.
- No protocol registration statement was detected.
-
