Segmenting Objects with Imbalanced Sizes via Smooth and Sparse Dual Optimal Transport

In image segmentation, object sizes (volumes) are often highly imbalanced, which adversely affects the performance of data-driven methods. To address this, we formulate the imbalance problem through optimal transport theory, providing a geometric interpretation of segmentation via Laguerre cell decomposition. By interpreting the dual variable of the volume constraint as a learnable network bias and solving the smooth semi-dual formulation iteratively while incorporating spatial information of pixels, we propose an iterative network-embedding layer, VP-Sparsemax, which enables end-to-end integration of volume priors into convolutional neural networks. Furthermore, we theoretically and experimentally demonstrate the critical role of sparsity, compared to traditional softmax or modified softmax, VP-Sparsemax better preserves volume in the segmentation results after argmax. We validated this performance with a toy example and four datasets including medical, autonomous driving and remote sensing images in three segmentation network baselines, achieving superior segmentation outcomes, particularly for small targets that are easily overlooked.