Development and evaluation of an artificial intelligence system for COVID-19 diagnosis

  1. Cheng Jin
  2. Weixiang Chen
  3. Yukun Cao
  4. Zhanwei Xu
  5. Zimeng Tan
  6. Xin Zhang
  7. Lei Deng
  8. Chuansheng Zheng
  9. Jie Zhou
  10. Heshui Shi
  11. Jianjiang Feng

Reviewed by

Read the full article See related articles

Listed in

Log in to save this article

Abstract

Early detection of COVID-19 based on chest CT enables timely treatment of patients and helps control the spread of the disease. We proposed an artificial intelligence (AI) system for rapid COVID-19 detection and performed extensive statistical analysis of CTs of COVID-19 based on the AI system. We developed and evaluated our system on a large dataset with more than 10 thousand CT volumes from COVID-19, influenza-A/B, non-viral community acquired pneumonia (CAP) and non-pneumonia subjects. In such a difficult multi-class diagnosis task, our deep convolutional neural network-based system is able to achieve an area under the receiver operating characteristic curve (AUC) of 97.81% for multi-way classification on test cohort of 3,199 scans, AUC of 92.99% and 93.25% on two publicly available datasets, CC-CCII and MosMedData respectively. In a reader study involving five radiologists, the AI system outperforms all of radiologists in more challenging tasks at a speed of two orders of magnitude above them. Diagnosis performance of chest x-ray (CXR) is compared to that of CT. Detailed interpretation of deep network is also performed to relate system outputs with CT presentations. The code is available at https://github.com/ChenWWWeixiang/diagnosis_covid19 .

Article activity feed

  1. ScreenIT

    SciScore for 10.1101/2020.03.20.20039834: (What is this?)

    Please note, not all rigor criteria are appropriate for all manuscripts.

    Table 1: Rigor

    NIH rigor criteria are not applicable to paper type.

    Table 2: Resources

    Software and Algorithms
    SentencesResources
    LASSO analysis was performed using python version 3.6 and the “ scikit-learn” package.
    python
    suggested: (IPython, RRID:SCR_001658)

    Results from OddPub: Thank you for sharing your code.

    Results from LimitationRecognizer: An explicit section about the limitations of the techniques employed in this study was not found. We encourage authors to address study limitations.

    Results from TrialIdentifier: No clinical trial numbers were referenced.

    Results from Barzooka: We did not find any issues relating to the usage of bar graphs.

    Results from JetFighter: We did not find any issues relating to colormaps.

    Results from rtransparent:
    • Thank you for including a conflict of interest statement. Authors are encouraged to include this statement when submitting to a journal.
    • Thank you for including a funding statement. Authors are encouraged to include this statement when submitting to a journal.
    • No protocol registration statement was detected.

    About SciScore

    SciScore is an automated tool that is designed to assist expert reviewers by finding and presenting formulaic information scattered throughout a paper in a standard, easy to digest format. SciScore checks for the presence and correctness of RRIDs (research resource identifiers), and for rigor criteria such as sex and investigator blinding. For details on the theoretical underpinning of rigor criteria and the tools shown here, including references cited, please follow this link.

  2. Version published to 10.1038/s41467-020-18685-1
  3. Version published to 10.1101/2020.03.20.20039834v3 on medRxiv
  4. Version published to 10.1101/2020.03.20.20039834v2 on medRxiv
  5. Version published to 10.1101/2020.03.20.20039834v1 on medRxiv