SCM: Semantic Segmentation with Dual-Stream Semantic Synergy under Adverse Weather Conditions

Unsupervised domain adaptation (UDA) aims to adapt a model trained on a source domain (\eg, clear weather data), to a target domain (\eg, adverse weather data), without the need for annotations in the target domain. Recent semantic segmentation models perform well under clear weather conditions but struggle with adverse weather conditions, which leading to poor segmentation results. There are two routes for these methods: designing a network for feature mapping or extracting domain-invariant features by additional modal information, but it is difficult to learn robust domain-invariant features. To address these issues, we introduce CLIP and design a dual-stream feature fusion network (DSFF), and Our DSFF allows CLIP and CNN encoders to benefit from each other, which increases model performance. This dual-stream design enables mutual enhancement between the two encoders, the frozen CLIP supplies the CNN encoder with high-level semantic cues relevant to the target domain, while the CNN encoder helps CLIP to reduce weather noise. Extensive experiments show that our method significantly outperforms other methods, achieving a mIoU of 59.0 on Cityscapes to ACDC.