COMPARISON OF ARTIFICIAL INTELLIGENCE ENABLED METHODS IN THE COMPUTED TOMOGRAPHIC ASSESSMENT OF COVID-19 DISEASE

  1. Atul Kapoor
  2. Goldaa Mahajan
  3. Aprajita Kapoor

Reviewed by

Read the full article See related articles

Listed in

Log in to save this article

Abstract

Objectives

Comparison of three different Artificial intelligence (AI) methods of assessment for patients undergoing Computed tomography (CT) for suspected Covid-19 disease. Parameters studied were probability of diagnosis, quantification of disease severity and the time to reach the diagnosis.

Methods

107 consecutive patients of suspected Covid-19 patients were evaluated using the three AI methods labeled as Al-I,II, III alongwith visual analysis labeled as VT for predicting probability of Covid-19, determining CT severity score (CTSS) and index (CTSI), percentage opacification (PO) and high opacification (POHO). Sensitivity, specificity along with area under curves were estimated for each method and the CTSS and CTSI correlated using Friedman test.

Results

Out of 107 patients 71 patients were Covid-19 positive and 20 negative by RT-PCR while 16 did not get RT-PCR done. Al-III method showed higher sensitivity and specificity of 93% and 88% respectively to predict probability of Covid 19. It had 2 false positive patients of interstitial lung disease. Al-II method had sensitivity and specificity of 66% and 83% respectively while visual (VT) analysis showed sensitivity and specificity of 59.7% and 62% respectively. Statistically significant differences were also seen in CTSI and PO estimation between Al-I and III methods (p< 0.0001) with Al-III showing fastest time to calculate results.

Conclusions

Al-III method gave better results to make an accurate and quick diagnosis of the Covid-19 with AUC of 0.85 to predict probability of Covid-19 alongwith quantification of Covid-19 lesions in the form of PO, POHO as compared to other AI methods and also by visual analysis.

KEY POINTS

CT examinations of the chest can be more accurate and informative in detecting Covid-19 if combined with AI methods which are being designed to achieve this objective. In this study we compared three AI methods with Visual analysis and the results show.

  • Al-III method had a higher sensitivity and specificity of 93% and 88% compared to other methods in predicting probability of Covid-19.

  • Significant inter method variations were seen in quantifying Covid-19 opacities as CTSS,CTSI, PO and POHO variables (p< 0.0001). Al-III method showed no statistical difference with VT method for PO variable (p = 0.24) and was the only method which depicted all the variables..

  • Time to processing results was the shortest with Al-III method.

Article activity feed

  1. ScreenIT

    SciScore for 10.1101/2020.09.02.20186650: (What is this?)

    Please note, not all rigor criteria are appropriate for all manuscripts.

    Table 1: Rigor

    Institutional Review Board StatementConsent: AI based methods using computed tomographic images of 107 consecutive patients suspected of having Covid-19 disease were included in the study after obtaining consent from local ethics review committee and informed consent from the patients.
    IRB: AI based methods using computed tomographic images of 107 consecutive patients suspected of having Covid-19 disease were included in the study after obtaining consent from local ethics review committee and informed consent from the patients.
    Randomizationnot detected.
    Blindingnot detected.
    Power Analysisnot detected.
    Sex as a biological variablenot detected.

    Table 2: Resources

    No key resources detected.

    Results from OddPub: We did not detect open data. We also did not detect open code. Researchers are encouraged to share open data when possible (see Nature blog).

    Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:
    The study also highlights limitations in various AI methods tested along with inter method variations of results like estimation of CTSI, percentage opacification and some more time may be required before it comes into clinical use. Out of the AI techniques compared Al-III method appears to be more advantageous and accurate compared with other AI methods including the visual method alone.

    Results from TrialIdentifier: No clinical trial numbers were referenced.

    Results from Barzooka: We did not find any issues relating to the usage of bar graphs.

    Results from JetFighter: We did not find any issues relating to colormaps.

    Results from rtransparent:
    • Thank you for including a conflict of interest statement. Authors are encouraged to include this statement when submitting to a journal.
    • Thank you for including a funding statement. Authors are encouraged to include this statement when submitting to a journal.
    • No protocol registration statement was detected.

    About SciScore

    SciScore is an automated tool that is designed to assist expert reviewers by finding and presenting formulaic information scattered throughout a paper in a standard, easy to digest format. SciScore checks for the presence and correctness of RRIDs (research resource identifiers), and for rigor criteria such as sex and investigator blinding. For details on the theoretical underpinning of rigor criteria and the tools shown here, including references cited, please follow this link.

  2. Version published to 10.1101/2020.09.02.20186650v1 on medRxiv