Enhancing Histological Cancer Cell Detection: Integrating XAI with Deep Learning for Improved Accuracy, Interpretability, and Clinical Trust

A precise and early histological diagnosis is critical for effective cancer treatment, reducing cancer-associated disability, and improving long-term quality of life. However, traditional deep learning (DL) models often function as black boxes, which limit their clinical adoption due to a lack of interpretability. To address this challenge, we propose a novel framework that integrates deep learning with Explainable Artificial Intelligence (XAI) to enhance both diagnostic accuracy and interpretability in histological cancer cell detection. The framework utilizes an ensemble of three pre-trained convolutional neural networks (CNNs): ResNet50, VGG16, and InceptionV3, combined with a 4x UltraSharp super-resolution technique to improve image clarity. XAI methods: Grad-CAM, SHAP, and LIME are incorporated to provide transparency into the model’s decision-making process. Our quantitative results show substantial improvements (1%-10%) in classification accuracy following upscaling, with ResNet50 achieving 96.21%, VGG16 92.14%, and InceptionV3 96.42%. This integrated approach addresses the challenge of achieving high accuracy on heterogeneous datasets where tissue sources are unspecified. Strong correlations were observed between the tumor regions identified by the models and the heatmaps generated by the XAI techniques. The framework’s interpretability and diagnostic performance were clinically validated by expert pathologists and further supported by a human trust survey involving 38 participants. This study demonstrates that integrating deep learning with XAI not only improves diagnostic performance but also fosters trust in AI-assisted digital pathology, paving the way for clinically reliable and interpretable AI systems in cancer detection.