Computer-aided covid-19 patient screening using chest images (X-Ray and CT scans)
This article has been Reviewed by the following groups
Listed in
- Evaluated articles (ScreenIT)
Abstract
Objectives
to evaluate the performance of Artificial Intelligence (AI) methods to detect covid-19 from chest images (X-Ray and CT scans).
Methods
Chest CT scans and X-Ray images collected from different centers and institutions were downloaded and combined together. Images were separated by patient and 66% of the patients were used to develop and train AI image-based classifiers. Then, the AI automated classifiers were evaluated on a separate set of patients (the remaining 33% patients).
Results (Chest X-Ray)
Five different data sources were combined for a total of N=9,841 patients (1,733 with covid-19, 810 with bacterial tuberculosis and 7,298 healthy patients). The test sample size was N=3,528 patients. The best AI method reached an Area Under the Curve (AUC) for covid-19 detection of 99%, with a detection rate of 96.4% at 1.0% false positive rate.
Results (Chest CT scans)
Two different data sources were combined for a total of N=363 patients (191 having covid-19 and 172 healthy patients). The test sample size was N=121 patients. The best AI method reached an AUC for covid-19 detection of 90.9%, with a detection rate of 90.6% at 24.6% false positive rate.
Conclusions
Computer aided automatic covid-19 detection from chest X-ray images showed promising results to be used as screening tool during the covid-19 outbreak. The developed method may help to manage patients better in case access to PCR testing is not possible or to detect patients with symptoms missed in a first round of PCR testing. The method will be made available online ( www.quantuscovid19.org ). These results merit further evaluation collecting more images. We hope this study will allow us to start such collaborations.
Article activity feed
-
-
SciScore for 10.1101/2020.07.16.20155093: (What is this?)
Please note, not all rigor criteria are appropriate for all manuscripts.
Table 1: Rigor
NIH rigor criteria are not applicable to paper type.Table 2: Resources
Software and Algorithms Sentences Resources All statistical analysis was then performed using Matlab (Mathworks, USA). Matlabsuggested: (MATLAB, RRID:SCR_001622)Results from OddPub: We did not detect open data. We also did not detect open code. Researchers are encouraged to share open data when possible (see Nature blog).
Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:Nevertheless, considering the many reported limitations of PCR discussed above, reducing false negatives in symptomatic patients through a fast and non-invasive procedure could be useful. Strengths: This …
SciScore for 10.1101/2020.07.16.20155093: (What is this?)
Please note, not all rigor criteria are appropriate for all manuscripts.
Table 1: Rigor
NIH rigor criteria are not applicable to paper type.Table 2: Resources
Software and Algorithms Sentences Resources All statistical analysis was then performed using Matlab (Mathworks, USA). Matlabsuggested: (MATLAB, RRID:SCR_001622)Results from OddPub: We did not detect open data. We also did not detect open code. Researchers are encouraged to share open data when possible (see Nature blog).
Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:Nevertheless, considering the many reported limitations of PCR discussed above, reducing false negatives in symptomatic patients through a fast and non-invasive procedure could be useful. Strengths: This study has several strengths. We have performed the study combining patients from several sources, each of which was in turn constructed using patients from different centers. This means that the data used is inherently multi-center and multiple equipment and operators were responsible for the acquisition of the images, giving more credibility to the results. Furthermore, we have evaluated the impact on performance of different patient’s training-validation splits and the relationship between patient’s variables and classifier’s mistakes. Moreover, we evaluated the classifier in a real transfer scenario, showing how performance dropped and by how much. Last but not least, for the first time as far as we know, we demonstrated that AI methods seem to be able to distinguish between covid-19 and bacterial infections (covid-19 vs bacterial pneumonias). Limitations: We acknowledge a number of limitations. We were not involved in data acquisition; we trusted the data as it came and weren’t able to double-check clinical outcomes of the images. However, these were public sources collected in large clinical institutions and widely used. Another limitation is the varying prevalence regarding covid-19 patients found in the datasets used, and the need to incorporate random controls from ot...
Results from TrialIdentifier: No clinical trial numbers were referenced.
Results from Barzooka: We did not find any issues relating to the usage of bar graphs.
Results from JetFighter: We did not find any issues relating to colormaps.
Results from rtransparent:- Thank you for including a conflict of interest statement. Authors are encouraged to include this statement when submitting to a journal.
- Thank you for including a funding statement. Authors are encouraged to include this statement when submitting to a journal.
- No protocol registration statement was detected.
-