Development and Validation of an Automated Radiomic CT Signature for Detecting COVID-19

  1. Julien Guiot
  2. Akshayaa Vaidyanathan
  3. Louis Deprez
  4. Fadila Zerka
  5. Denis Danthine
  6. Anne-Noëlle Frix
  7. Marie Thys
  8. Monique Henket
  9. Gregory Canivet
  10. Stephane Mathieu
  11. Evanthia Eftaxia
  12. Philippe Lambin
  13. Nathan Tsoutzidis
  14. Benjamin Miraglio
  15. Sean Walsh
  16. Michel Moutschen
  17. Renaud Louis
  18. Paul Meunier
  19. Wim Vos
  20. Ralph T. H. Leijenaar
  21. Pierre Lovinfosse

Reviewed by

Read the full article

Discuss this preprint

Start a discussion What are Sciety discussions?

Listed in

Log in to save this article

Abstract

The coronavirus disease 2019 (COVID-19) outbreak has reached pandemic status. Drastic measures of social distancing are enforced in society and healthcare systems are being pushed to and beyond their limits. To help in the fight against this threat on human health, a fully automated AI framework was developed to extract radiomics features from volumetric chest computed tomography (CT) exams. The detection model was developed on a dataset of 1381 patients (181 COVID-19 patients plus 1200 non COVID control patients). A second, independent dataset of 197 RT-PCR confirmed COVID-19 patients and 500 control patients was used to assess the performance of the model. Diagnostic performance was assessed by the area under the receiver operating characteristic curve (AUC). The model had an AUC of 0.882 (95% CI: 0.851–0.913) in the independent test dataset (641 patients). The optimal decision threshold, considering the cost of false negatives twice as high as the cost of false positives, resulted in an accuracy of 85.18%, a sensitivity of 69.52%, a specificity of 91.63%, a negative predictive value (NPV) of 94.46% and a positive predictive value (PPV) of 59.44%. Benchmarked against RT-PCR confirmed cases of COVID-19, our AI framework can accurately differentiate COVID-19 from routine clinical conditions in a fully automated fashion. Thus, providing rapid accurate diagnosis in patients suspected of COVID-19 infection, facilitating the timely implementation of isolation procedures and early intervention.

Article activity feed

  1. Version published to 10.3390/diagnostics11010041
  2. ScreenIT

    SciScore for 10.1101/2020.04.28.20082966: (What is this?)

    Please note, not all rigor criteria are appropriate for all manuscripts.

    Table 1: Rigor

    Institutional Review Board StatementIRB: Ethics: The study has been approved by the local ethics committee of the CHU-Liège (EC number 116/2020).
    Consent: The institutional review board waived the requirement to obtain written informed consent for this retrospective case series, since all analyses were performed on de-identified (i.e., anonymized) data and there was no potential risk to patients.
    RandomizationStratified partitioning was performed to randomly split the data into 80% for model training and 20% for validation.
    Blindingnot detected.
    Power Analysisnot detected.
    Sex as a biological variablenot detected.

    Table 2: Resources

    No key resources detected.

    Results from OddPub: We did not detect open data. We also did not detect open code. Researchers are encouraged to share open data when possible (see Nature blog).

    Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:
    We have encountered some limitations in this study. Firstly, COVID-19 is caused by a SARS-CoV-2 and may have similar imaging characteristics as pneumonia caused by other types of viruses. However, due to the lack of laboratory confirmation of the aetiology for each of these cases, we were not able to select other viral pneumonias for comparison in this study. Although our Control group of non-COVID-19 patients contains several patients (CAP in Table 1, 12.5%) with pneumonia (either viral, bacterial or organizing pneumonia from any cause), it would be desirable to test the performance of our algorithm in distinguishing COVID-19 from other viral pneumonias that have real time polymerase chain reaction confirmation of the viral agent in a future study. Furthermore, there is a large commonality in how the lung responds to various pathologies and there is a considerable amount of similarity in the presentation of many diseases in the lung that depend on a plethora of factors (e.g. age, drug reactivity, immune status, underlying co-morbidities). It is likely impossible to differentiate all lung diseases by way of imaging and AI alone, and as such, drastic improvements in positive predictive value are likely difficult to achieve. A multi-omics approach for this is most likely optimal. Future work is planned to collect additional chest CTs from multiple centres to externally validate the performance of our algorithm. Ultimately, this study focuses on diagnosis whereas prognosis on th...

    Results from TrialIdentifier: No clinical trial numbers were referenced.

    Results from Barzooka: We did not find any issues relating to the usage of bar graphs.

    Results from JetFighter: We did not find any issues relating to colormaps.

    Results from rtransparent:
    • Thank you for including a conflict of interest statement. Authors are encouraged to include this statement when submitting to a journal.
    • Thank you for including a funding statement. Authors are encouraged to include this statement when submitting to a journal.
    • No protocol registration statement was detected.

    About SciScore

    SciScore is an automated tool that is designed to assist expert reviewers by finding and presenting formulaic information scattered throughout a paper in a standard, easy to digest format. SciScore checks for the presence and correctness of RRIDs (research resource identifiers), and for rigor criteria such as sex and investigator blinding. For details on the theoretical underpinning of rigor criteria and the tools shown here, including references cited, please follow this link.

  3. Version published to 10.1101/2020.04.28.20082966 on medRxiv