AI/ML Models to Aid in the Diagnosis of COVID-19 Illness from Forced Cough Vocalizations: Results and Challenges of a Systematic Review of the Relevant Literature

  1. K. Kelley
  2. A.A. Sakara
  3. M. Kelley
  4. S. C. Kelley
  5. P. McLenaghan
  6. R. Aldir
  7. M. Cox
  8. N. Donaldson
  9. A. Stogsdill
  10. S. Kotchou
  11. G. Sula
  12. M.A. Ramirez

Reviewed by

Read the full article See related articles

Listed in

Log in to save this article

Abstract

From a comprehensive and systematic search of the relevant literature on signal data signature (SDS)-based artificial intelligence/machine learning (AI/ML) systems designed to aid in the diagnosis of COVID-19 illness, we aimed to reproduce the reported systems and to derive a performance goal for comparison to our own medical device with the same intended use. These objectives were in line with a pathway to regulatory approval of such devices, as well as to acceptance of this unfamiliar technology by disaster/pandemic decision makers and clinicians. To our surprise, none of the peer-reviewed articles or pre-print server records contained details sufficient to meet the planned objectives. Information amassed from the full review of more than 60 publications, however, did underscore discrete impediments to bringing AI/ML diagnostic solutions to the bedside during a pandemic. These challenges then were explored by the authors via a gap analysis and specific remedies were proposed for bringing AI/ML technologies in closer alignment with the needs of a Total Product Life Cycle (TPLC) regulatory approach.

Article activity feed

  1. ScreenIT

    SciScore for 10.1101/2021.11.12.21266271: (What is this?)

    Please note, not all rigor criteria are appropriate for all manuscripts.

    Table 1: Rigor

    Ethicsnot detected.
    Sex as a biological variablenot detected.
    Randomizationnot detected.
    Blindingnot detected.
    Power Analysisnot detected.

    Table 2: Resources

    Software and Algorithms
    SentencesResources
    EndNote 2020 was the designated reference manager and PubMed was searched via this software.
    EndNote
    suggested: (EndNote, RRID:SCR_014001)
    PubMed
    suggested: (PubMed, RRID:SCR_004846)
    Serial searches of “Any Field” in PubMed, “Full Text and Metadata” in the IEEE Xplore digital library of the Institute of Electrical and Electronics Engineers,(ieeexplore.ieee.org), “All Fields” in the arXiv open-access archive (arxiv.org), and “Full Text or Abstract or Title” in bioRxiv and medRxiv (medrxiv.org) were performed using the identical search terms as listed below: The results from these serial searches were combined and systematically filtered to achieve a final article pool from which all references would be evaluated for contribution to the stated objectives.
    Metadata”
    suggested: None
    arXiv
    suggested: (arXiv, RRID:SCR_006500)
    bioRxiv
    suggested: (bioRxiv, RRID:SCR_003933)

    Results from OddPub: We did not detect open data. We also did not detect open code. Researchers are encouraged to share open data when possible (see Nature blog).

    Results from LimitationRecognizer: An explicit section about the limitations of the techniques employed in this study was not found. We encourage authors to address study limitations.

    Results from TrialIdentifier: No clinical trial numbers were referenced.

    Results from Barzooka: We did not find any issues relating to the usage of bar graphs.

    Results from JetFighter: We did not find any issues relating to colormaps.

    Results from rtransparent:
    • Thank you for including a conflict of interest statement. Authors are encouraged to include this statement when submitting to a journal.
    • Thank you for including a funding statement. Authors are encouraged to include this statement when submitting to a journal.
    • No protocol registration statement was detected.

    Results from scite Reference Check: We found no unreliable references.

    About SciScore

    SciScore is an automated tool that is designed to assist expert reviewers by finding and presenting formulaic information scattered throughout a paper in a standard, easy to digest format. SciScore checks for the presence and correctness of RRIDs (research resource identifiers), and for rigor criteria such as sex and investigator blinding. For details on the theoretical underpinning of rigor criteria and the tools shown here, including references cited, please follow this link.

  2. Version published to 10.1101/2021.11.12.21266271v1 on medRxiv