UnionFusion: Improving Misaligned Multi-modality Image Registration And Fusion Via Feature Enhancement

Image registration and fusion aim to align multi-modality images, generating fused image with richer information and higher quality.Existing fusion methods correct spatial misalignment through geometric transformation, semantic guidance, and cross-modality complementarity.However, these methods adopt fixed receptive fields, overlooking local fine-grained features, which leads to structural distortions and edge artifacts.To address these issues, this paper proposes an improved approach for misaligned multi-modality image registration and fusion via feature enhancement, called UnionFusion.Firstly, to achieve the elastic receptive field in registration and fusion tasks, we develop Separable Deformable Convolution (SDConv), which utilizes group-specific learnable sampling offsets to enhance feature representation.Secondly, to address image spatial misalignment, we design Multi-scale Deformable Adaptive Registration module (MDAR) toestimate the deformation field between the source and target images, correcting geometric distortions.Thirdly, to enhance fine-grained features, we design differential fusion mechanism, Shallow Deformable Fusion (SDF) to extract shallow features and Deep Progressive Fusion (DPF) to capture deep hierarchical features.Finally, to enhance the complementarity of the joint image registration and fusion, we employ symmetric network and symmetric loss.Extensive experimental analysis demonstrates the superiority of the proposed method in three multi-modality image fusion datasets.