EMSA-Net: Enhancing Brain Tumor Segmentation with Equivalent Multi-Scale and Cross-Modality Attention under limited MRI Modalities

Multi-modality medical imaging is essential for disease diagnosis and clinical treatment as it can provide complementary information on the morphological features of tumors. Clinically, acquiring four complete modalities is difficult due to device limitations, scan time, and limited costs, such that getting high segmentation performance for brain tumors becomes more challenging. Scale Module (EMS) utilizes the available modalities to extract brain tumor regions fully and integrates features according to the contributions of available modalities. Cross-Modality Attention Mechanism (CMA) further fuses multi-modality features and mitigates the impact of limited modalities in medical image segmentation. In this way, EMSA-Net can effectively capture the relationship between different modalities and tumor regions, enhancing the robustness of the model. Additionally, this paper introduces a knowledge distillation method where the student network is trained on a subset of the teacher’s inputs. This approach allows the student model to learn from the teacher model’s knowledge, enabling effective segmentation even with limited modalities. The proposed method is evaluated on the BraTS2020 dataset, demonstrating its effectiveness in improving segmentation performance under limited modality conditions. The performance of the proposed model is superior to state-of-the-art methods. For all 15 multi-modality combinations, the proposed method achieves average Dice scores of 88.7% for WT, 82.7% for TC, and 68.8% for ET, outperforming other methods by 0.8%, 2.9%, and 2.1% respectively. More important, limited modalities, such as T1ce, also perform comparably to complete modalities.