Curbing the AI-induced enthusiasm in diagnosing COVID-19 on chest X-Rays: the present and the near-future

  1. Alexandru Burlacu
  2. Radu Crisan-Dabija
  3. Iolanda Valentina Popa
  4. Bogdan Artene
  5. Vasile Birzu
  6. Mihai Pricop
  7. Cristina Plesoianu
  8. Daniele Generali

Reviewed by

Read the full article

Listed in

Log in to save this article

Abstract

In the current context of COVID-19 pandemic, a rapid and accessible screening tool based on image processing of chest X-rays (CXRs) using machine learning (ML) approaches would be much needed. Initially, we intended to create and validate an ML software solution able to discriminate on the basis of the CXR between SARS-CoV-2-induced bronchopneumonia and other bronchopneumonia etiologies.

A systematic search of PubMed, Scopus and arXiv databases using the following search terms [“artificial intelligence” OR “deep learning” OR “neural networks”], AND [“COVID-19” OR “SARS-CoV-2”] AND [“chest X-ray” OR “CXR” OR “X-ray”] found 14 recent studies. Most of them declared to be able to confidently identify COVID-19 based on CXRs using deep neural networks. Firstly, weaknesses of artificial intelligence (AI) solutions were analyzed, tackling the issues with datasets (from both medical and technical points of view) and the vulnerability of used algorithms. Then, arguments were provided for why our study design is stronger and more realistic than the previously quoted papers, balancing the possible false expectations with facts.

The authors consider that the potential of AI use in COVID-19 diagnosis on CXR is real. However, scientific community should be careful in interpreting statements, results and conclusions regarding AI use in imaging. It is therefore necessary to adopt standards for research and publication of data, because it seems that in the recent months scientific reality suffered manipulations and distortions. Also, a call for responsible approaches to the imaging methods in COVID-19 is raised. It seems mandatory to follow some rigorous approaches in order to provide with adequate results in daily routine. In addition, the authors intended to raise public awareness about the quality of AI protocols and algorithms and to encourage public sharing of as many CXR images with common quality standards.

Article activity feed

  1. ScreenIT

    SciScore for 10.1101/2020.04.28.20082776: (What is this?)

    Please note, not all rigor criteria are appropriate for all manuscripts.

    Table 1: Rigor

    Institutional Review Board Statementnot detected.
    Randomizationnot detected.
    Blindingnot detected.
    Power Analysisnot detected.
    Sex as a biological variablenot detected.

    Table 2: Resources

    Software and Algorithms
    SentencesResources
    For this purpose, the electronic databases of PubMed, Scopus and arXiv were systematically searched for relevant articles from the inception until late March 2020.
    PubMed
    suggested: (PubMed, RRID:SCR_004846)

    Results from OddPub: We did not detect open data. We also did not detect open code. Researchers are encouraged to share open data when possible (see Nature blog).

    Results from LimitationRecognizer: An explicit section about the limitations of the techniques employed in this study was not found. We encourage authors to address study limitations.

    Results from TrialIdentifier: We found the following clinical trial numbers in your paper:

    IdentifierStatusTitle
    NCT04313946RecruitingArtificial Intelligence Algorithms for Discriminating Betwee…

    Results from Barzooka: We did not find any issues relating to the usage of bar graphs.

    Results from JetFighter: We did not find any issues relating to colormaps.

    Results from rtransparent:
    • Thank you for including a conflict of interest statement. Authors are encouraged to include this statement when submitting to a journal.
    • Thank you for including a funding statement. Authors are encouraged to include this statement when submitting to a journal.
    • No protocol registration statement was detected.

    About SciScore

    SciScore is an automated tool that is designed to assist expert reviewers by finding and presenting formulaic information scattered throughout a paper in a standard, easy to digest format. SciScore checks for the presence and correctness of RRIDs (research resource identifiers), and for rigor criteria such as sex and investigator blinding. For details on the theoretical underpinning of rigor criteria and the tools shown here, including references cited, please follow this link.

  2. Version published to 10.1101/2020.04.28.20082776v1 on medRxiv