A machine-learning algorithm for diagnosis of multisystem inflammatory syndrome in children and Kawasaki disease in the USA: a retrospective model development and validation study
This article has been Reviewed by the following groups
Listed in
- Evaluated articles (ScreenIT)
Abstract
No abstract available
Article activity feed
-
-
SciScore for 10.1101/2022.02.07.21268280: (What is this?)
Please note, not all rigor criteria are appropriate for all manuscripts.
Table 1: Rigor
Ethics Consent: Written consent or assent as appropriate was obtained from parents and subjects and the study was approved by the Institutional Review Boards (IRB) of the University of California San Diego (UCSD), Connecticut Children’s Medical Center, Children’s Hospital Los Angeles, Boston Children’s Hospital, and Children’s National Hospital.
IRB: Written consent or assent as appropriate was obtained from parents and subjects and the study was approved by the Institutional Review Boards (IRB) of the University of California San Diego (UCSD), Connecticut Children’s Medical Center, Children’s Hospital Los Angeles, Boston Children’s Hospital, and Children’s National Hospital.Sex as a biological … SciScore for 10.1101/2022.02.07.21268280: (What is this?)
Please note, not all rigor criteria are appropriate for all manuscripts.
Table 1: Rigor
Ethics Consent: Written consent or assent as appropriate was obtained from parents and subjects and the study was approved by the Institutional Review Boards (IRB) of the University of California San Diego (UCSD), Connecticut Children’s Medical Center, Children’s Hospital Los Angeles, Boston Children’s Hospital, and Children’s National Hospital.
IRB: Written consent or assent as appropriate was obtained from parents and subjects and the study was approved by the Institutional Review Boards (IRB) of the University of California San Diego (UCSD), Connecticut Children’s Medical Center, Children’s Hospital Los Angeles, Boston Children’s Hospital, and Children’s National Hospital.Sex as a biological variable not detected. Randomization Finally, 200 randomly sampled patients were used for the FC and KD trust sets, and all MIS-C patients were used for the MIS-C trust set. Blinding not detected. Power Analysis not detected. Table 2: Resources
Software and Algorithms Sentences Resources KD patients met the case definition of the American Heart Association18 for either complete or incomplete KD. American Heartsuggested: (American Heart Association, RRID:SCR_007210)2.3.1 with a logistic regression using scikit-learn v0.24.2 as the baseline model. scikit-learnsuggested: (scikit-learn, RRID:SCR_002577)Shapley Values: To explain the model predictions, we calculated the Shapley values for the test set using the SHAP Python library13. Pythonsuggested: (IPython, RRID:SCR_001658)Results from OddPub: We did not detect open data. We also did not detect open code. Researchers are encouraged to share open data when possible (see Nature blog).
Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:We recognize limitations to our work due to the limited availability of FC and KD data for external validation. There is no gold standard for KD or MIS-C diagnosis. Thus, we cannot exclude some degree of misdiagnosis in either the training or test set. However, the validation performance demonstrated internal consistency with the signs and laboratory tests used by KIDMATCH. It is unknown how the model would perform on FC and KD patients from other hospitals as we were unable to obtain these data. The thresholds established during internal validation may not be generalizable to different sites and shifting the threshold may be required to adjust for different prevalence rates. However, the high model AUC means that the model can be used to effectively prioritize febrile patients for further evaluation of MIS-C or KD. A key step for deployment will be to establish standardized conditions for use so the algorithm is applied to the appropriate patients. The current algorithm is only optimized for laboratory test values collected at the time of initial evaluation, and it is unknown how it would perform with data collected at a later timepoint. It is also unknown how end users should deal with patients flagged as indeterminate, but a possible solution could be to order more specialized tests such as ferritin, troponin, BNP/NT-proBNP, and D-dimer as well as IgG antibody to SARS-CoV-2 as is routine practice for suspected MIS-C patients16. Based on these data, the proposed algorithm i...
Results from TrialIdentifier: No clinical trial numbers were referenced.
Results from Barzooka: We did not find any issues relating to the usage of bar graphs.
Results from JetFighter: We did not find any issues relating to colormaps.
Results from rtransparent:- Thank you for including a conflict of interest statement. Authors are encouraged to include this statement when submitting to a journal.
- Thank you for including a funding statement. Authors are encouraged to include this statement when submitting to a journal.
- No protocol registration statement was detected.
Results from scite Reference Check: We found no unreliable references.
-
