SpectraViT: A Novel Hybrid Architecture for Enhanced Melanoma Classification

Melanoma is one of the most invasive types of skin cancers which requires an accurate understanding of the nature of the disease for effective management. However, subtle differences in lesion morphology are one of the greatest challenges for traditional methods. This research presents SpectraViT, a hybrid architecture that combines Fourier and Wavelet feature extraction techniques as part of a Vision Transformer (ViT) model. This approach transforms image structures, captures both spatial and frequency information, and enhances feature representation, enabling the model to interpret intricate patterns in skin lesions. In this paper we have discussed our experimental findings which have shown improvements in accuracy as compared to the existing techniques like CNNs, ViT, Fourier ViT, and Wavelet ViT on the same dataset, thus validating the applicability of our framework for reliable melanoma diagnosis. We evaluate SpectraViT on a diverse melanoma dataset and demonstrate significant improvements in classification accuracy, achieving over 91% on test data.