Adaptive Heterogeneous Representation Learning for Reliable and Compact Pap Smear Screening

Cervical cancer remains a major cause of preventable mortality among women, making reliable cytology screening essential for early detection. However, manual Pap smear assessment is labour-intensive, time-consuming, and susceptible to inter-observer variability, while automated analysis is challenged by subtle inter-class differences, pronounced intra-class heterogeneity, and staining variability. This study presents a reliable and compact deep learning framework for automated Pap smear screening that addresses these challenges through adaptive fusion of complementary cytomorphological representations. The proposed model combines ResNet-50 and ConvNeXt-Base within a Dual Attention Fusion mechanism that assigns sample-specific importance to heterogeneous feature streams, enabling more discriminative integration than fixed fusion strategies. To improve efficiency, a Hierarchical Autoencoder learns compact task-relevant latent representations while preserving diagnostic performance. Evaluated on the SIPaKMeD and Herlev datasets under multiclass and binary screening settings using five-fold cross-validation, the framework achieves macro-F1 scores of 99.66% and 98.15%, with binary F1 scores of 99.65% and 99.41%, respectively. McNemar’s test indicates significant gains over single-backbone and non-adaptive fusion baselines. Grad-CAM highlights diagnostically relevant nuclear and peri-nuclear regions, while cross-dataset experiments suggest improved robustness under domain shift. These findings support the proposed framework as an interpretable and parameter-efficient approach for automated cervical cytology screening.