MSNet: A novel network with comprehensive multi-scale feature integration for gastric cancer and colon polyp segmentation

The precise and reliable segmentation of endoscopic images plays a pivotal role in the early diagnosis of gastrointestinal cancer lesions like gastric cancer and colon polyp. Due to the variations in appearance and blurred boundaries of cancer lesions, achieving precise automatic segmentation remains challenging. Therefore, this paper proposes a novel CNN-Transformer hybrid network named MSNet, which comprehensively utilizes multi-scale features. It aims to improve the segmentation accuracy of gastrointestinal cancer lesions by enhancing the ability to capture multi-scale features and distinguish boundaries. Firstly, a multi-scale perception module (MSPM) is designed specifically to capture and enhance multi-scale features, employing multiple pathways with stripe convolutions of various kernel sizes. Secondly, to enhance the model's ability to identify target boundaries, an improved dual feature pyramid (IDFP) strategy is proposed to obtain multi-scale prediction maps from various pathways and stages, thereby aggregating multi-scale features. Thirdly, to refine the network's recognition of target boundaries, a Boundary Enhancement Module (BEM) is devised, which integrates the multi-scale prediction maps obtained from IDFP with the final output of the decoder. This integration intends to mitigate spatial information loss caused by consecutive downsampling and upsampling operations, thereby achieving more precise segmentation outcomes. Extensive experiments are conducted on the privately collected gastric endoscopy dataset and five publicly available colonoscopy polyp datasets, aiming to assess the effectiveness and generalization of the proposed method. Results demonstrate that the proposed network achieves superior performance compared to other state-of-the-art methods in terms of accuracy. The source code will be available at https://github.com/0xChunFeng/MSNet.