Multi-Modal Ancient Script Recognition via deep learning with Data Homogenization and Augmentation

Ancient scripts provide invaluable insights into ancient societies, and their effective recognition is crucial for cultural relic preservation, textual decipherment, and heritage. Current research primarily focuses on single mode ancient text data recognition such as processing rubbings or handwritten scripts independently, yet ancient scripts exhibit diverse forms across modalities. To address this, we propose a novel multimodal recognition framework capable of processing hybrid inputs like oracle bone rubbings and handwritten scripts. Our method employs two additional modules, a cross-modal data homogenization block‌ to unify heterogeneous data representations and ‌a data augmentation block‌ to enhance model robustness, then achieve the recognition with convolutional neural networks. Evaluated on oracle bone and bronze inscription datasets, our approach outperforms baseline methods in recognition accuracy and generalization capability across modalities.