Label-Graph Guided Semantic Alignment for Multi-Class Remote Sensing Image Recognition

In multi-class remote sensing image classification, semantic confusion among similar classes often limits performance, as conventional one-hot labels cannot express inter-class relationships. To address this issue, we propose a Label-Graph-Guided Semantic Alignment method that leverages a data-driven label graph to enhance both the training supervision and the model predictions. Specifically, we construct a label graph from the confusion matrix to capture class similarities, and introduce two novel mechanisms: Graph-Soft at the supervision level and Graph-Logit at the prediction level. Graph-Soft utilizes the label graph to refine one-hot labels into soft label vectors. Meanwhile, Graph-Logit encourages the model's logits to respect label similarities, aligning the semantic space of predictions with that of labels. The proposed approach is lightweight and architecture-agnostic, introducing negligible computational overhead and requiring no additional annotations. Experiments on multiple remote sensing scene classification benchmarks (EuroSAT, MLRSNet and RESISC45) with different backbones (ResNet18, ResNet50) demonstrate consistent performance gains. Our method outperforms standard one-hot training across all tested datasets, improving classification accuracy, AUC, and F1-score. Notably, it achieves up to a 2-3% increase in accuracy on challenging benchmarks compared to the baseline. These results validate that incorporating label graph knowledge effectively reduces semantic confusion and enhances multi-class classification performance. The source code is publicly available at https://github.com/zhoukuniyc/GLSGLR/tree/master.