A Deep Learning Model for Multiclass Lung Cancer Classification Using Multimodal Data Fusion

Background: Lung cancer is the most prevalent malignant tumor in terms of morbidity and mortality. Accurate classification and risk staging are essential for developing appropriate treatment plans and achieving precision in lung cancer diagnosis and treatment. Methods: A lung cancer pathology image classification model was trained using the ResNet152 architecture, which supports custom input image block sizes and serves as a feature extractor. The MMLM integrates pathological images, RNA sequencing data, methylation microarray data, and clinical information through a feature-level fusion approach to emulate the comprehensive judgment process of pathologists. Class activation mapping (CAM) and UMAP mapping were employed to provide intuitive visual representations of the model's decision-making process. Results: The integration of multimodal data resulted in a notable enhancement in the accuracy of lung cancer histological classification and risk prediction. The use of CAM and UMAP mapping improved the interpretability of the model, fostering greater trust among medical professionals in its decision-making capabilities. Conclusions: The proposed MMLM significantly advances lung cancer diagnosis by integrating diverse data sources, enhancing both accuracy and interpretability. Future integration of additional modal data will further optimize the model's comprehensiveness and accuracy in clinical applications, offering a powerful tool for personalized diagnosis and treatment planning.