A Transformer-based Multi-label Defect Image Classification Algorithm with Label Correlation Fusion

The structural quality and performance safety of urban infrastructure, such as sewer pipes and bridges, are of paramount importance. However, due to the complex structure and unique locations of these facilities, traditional manual inspection methods are not only inefficient but also pose significant safety risks. Therefore, developing automated defect detection and classification technologies is of great significance for improving the efficiency and safety of infrastructure maintenance. This paper proposes a multi-label defect image classification algorithm based on Transformer networks and label correlation fusion (LabelMDIC), aiming to address the limitations of existing methods in utilizing label co-occurrence relationships and fusing visual features across different scales in building facility defect detection. LabelMDIC employs the Swin Transformer to extract multi-scale defect features and leverages the self-attention mechanism to model the contextual semantics and co-occurrence relationships of defect labels, thereby supervising the visual feature reasoning process of defect images. Additionally, the algorithm integrates the co-occurrence relationship features of defect labels at different stages of the network to enhance the expression of information during the feature extraction process and introduces an Asymmetric Loss function to improve the model's ability to learn positive labels while reducing the impact of negative labels. Comparative experimental results on multiple datasets demonstrate that the LabelMDIC model not only achieves excellent classification accuracy but also shows significant advantages in terms of model complexity and inference speed, providing an efficient and practical solution for multi-label defect image classification tasks in building facilities.