Enhancing Early Skin Cancer Detection: A Deep Learning Approach with Multi-Scale Feature Refinement and Fusion

The global incidence of skin cancer is rising, making it an increasingly critical public health issue. Malignant skin tumors such as melanoma originate from pathological alterations in skin cells, and their accurate early-stage segmentation is crucial for quantitative analysis, early diagnosis, and effective treatment. However, achieving precise and efficient segmentation remains a major challenge, as existing methods often struggle to capture complex lesion characteristics. To address this challenge, we propose a novel deep learning framework that integrates the PVT v2 backbone with two key modules: the Spatial-Aware Feature Enhancement (SAFE) module and the Multiscale Dual Cross-attention Fusion (MDCF) module. The SAFE module enhances multi-scale encoder features through a dual-branch architecture, which adaptively extracts offset information to integrate fine-grained shallow details with deep semantic information, thereby bridging the feature gap across network depths. The MDCF module establishes bidirectional cross-attention between decoder and encoder features, followed by multi-scale deformable convolutions that capture lesion boundaries and small fragments across heterogeneous receptive fields, thereby enriching semantic details while suppressing background interference. The proposed model was evaluated on two public benchmark datasets (ISIC 2016 and ISIC 2018), achieving Intersection over Union (IoU) scores of 87.33% and 83.67%, respectively. These results demonstrate superior performance compared to current state-of-the-art methods and indicate that our framework significantly enhances skin lesion image analysis, offering a promising tool for improving early detection of skin cancer.