Explainable Multimodal AI for In-Hospital Detection of Lung Cancer and Heart Failure Using Routine Data: Development, Internal Validation, and Simulated Federated Evaluation
Listed in
This article is not in any list yet, why not save it to one of your lists.Abstract
Objective
To determine whether combining four routine data sources improves in-hospital detection of lung cancer and heart failure, which sources matter, whether explanations are faithful, whether models are fair, and how much benefit federated training retains.
Methods
This retrospective single-centre study linked MIMIC-IV admissions (73,637 patients) to chest radiographs, radiology reports, electrocardiograms (ECGs) and laboratory tests from the first 48 hours. Excluding prior cases, we compared single-source models, gradient boosting and early, late and attention-based fusion, tested explanations for faithfulness and weight dependence, audited subgroup performance, and simulated five hospitals for federated training.
Results
Lung cancer was coded in 1.7% and heart failure in 15.6%. Attention fusion reached an area under the receiver operating characteristic curve (AUROC) of 0.833 (95% CI 0.809–0.859) for lung cancer and 0.885 (0.878–0.892) for heart failure, against 0.753 and 0.794 for gradient boosting on laboratory data (both p < 0.001); early and late fusion performed similarly. Lung cancer relied most on laboratory data and reports, heart failure on the ECG. Explanations passed a deletion test; weight randomisation of absolute attributions passed for ECG but not laboratory explanations. Heart-failure discrimination fell with age. Federated training recovered 95% of the centralised gain over local training for heart failure; for lung cancer, federated averaging (FedAvg) recovered 51% and FedProx 82%.
Conclusion
Fusing routine data improved in-hospital detection of both diseases over laboratory data alone, and which data matter depends on the disease. Explanations should be checked per source, and performance in older patients monitored. External validation is needed.
