Curbing the AI-induced enthusiasm in diagnosing COVID-19 on chest X-Rays: the present and the near-future
This article has been Reviewed by the following groups
Listed in
- Evaluated articles (ScreenIT)
Abstract
In the current context of COVID-19 pandemic, a rapid and accessible screening tool based on image processing of chest X-rays (CXRs) using machine learning (ML) approaches would be much needed. Initially, we intended to create and validate an ML software solution able to discriminate on the basis of the CXR between SARS-CoV-2-induced bronchopneumonia and other bronchopneumonia etiologies.
A systematic search of PubMed, Scopus and arXiv databases using the following search terms [“artificial intelligence” OR “deep learning” OR “neural networks”], AND [“COVID-19” OR “SARS-CoV-2”] AND [“chest X-ray” OR “CXR” OR “X-ray”] found 14 recent studies. Most of them declared to be able to confidently identify COVID-19 based on CXRs using deep neural networks. Firstly, weaknesses of artificial intelligence (AI) solutions were analyzed, tackling the issues with datasets (from both medical and technical points of view) and the vulnerability of used algorithms. Then, arguments were provided for why our study design is stronger and more realistic than the previously quoted papers, balancing the possible false expectations with facts.
The authors consider that the potential of AI use in COVID-19 diagnosis on CXR is real. However, scientific community should be careful in interpreting statements, results and conclusions regarding AI use in imaging. It is therefore necessary to adopt standards for research and publication of data, because it seems that in the recent months scientific reality suffered manipulations and distortions. Also, a call for responsible approaches to the imaging methods in COVID-19 is raised. It seems mandatory to follow some rigorous approaches in order to provide with adequate results in daily routine. In addition, the authors intended to raise public awareness about the quality of AI protocols and algorithms and to encourage public sharing of as many CXR images with common quality standards.
Article activity feed
-
SciScore for 10.1101/2020.04.28.20082776: (What is this?)
Please note, not all rigor criteria are appropriate for all manuscripts.
Table 1: Rigor
Institutional Review Board Statement not detected. Randomization not detected. Blinding not detected. Power Analysis not detected. Sex as a biological variable not detected. Table 2: Resources
Software and Algorithms Sentences Resources For this purpose, the electronic databases of PubMed, Scopus and arXiv were systematically searched for relevant articles from the inception until late March 2020. PubMedsuggested: (PubMed, RRID:SCR_004846)Results from OddPub: We did not detect open data. We also did not detect open code. Researchers are encouraged to share open data when possible (see Nature blog).
Results from LimitationRecognizer: An explicit section about the limitations of the techniques employed in this …SciScore for 10.1101/2020.04.28.20082776: (What is this?)
Please note, not all rigor criteria are appropriate for all manuscripts.
Table 1: Rigor
Institutional Review Board Statement not detected. Randomization not detected. Blinding not detected. Power Analysis not detected. Sex as a biological variable not detected. Table 2: Resources
Software and Algorithms Sentences Resources For this purpose, the electronic databases of PubMed, Scopus and arXiv were systematically searched for relevant articles from the inception until late March 2020. PubMedsuggested: (PubMed, RRID:SCR_004846)Results from OddPub: We did not detect open data. We also did not detect open code. Researchers are encouraged to share open data when possible (see Nature blog).
Results from LimitationRecognizer: An explicit section about the limitations of the techniques employed in this study was not found. We encourage authors to address study limitations.Results from TrialIdentifier: We found the following clinical trial numbers in your paper:
Identifier Status Title NCT04313946 Recruiting Artificial Intelligence Algorithms for Discriminating Betwee… Results from Barzooka: We did not find any issues relating to the usage of bar graphs.
Results from JetFighter: We did not find any issues relating to colormaps.
Results from rtransparent:- Thank you for including a conflict of interest statement. Authors are encouraged to include this statement when submitting to a journal.
- Thank you for including a funding statement. Authors are encouraged to include this statement when submitting to a journal.
- No protocol registration statement was detected.
-