Semantic-Aware Decoding of Covert Inner Speech: A Multimodal EEG–EMG–Audio Framework

Non-invasive brain–computer interfaces (BCIs) aim to restore communication by decoding intended messages directly from neural activity, even when no audible speech is produced. However, evidence remains limited that non-invasive signals can support semantic-level decoding of covert (inner) speech under subject-held-out evaluation. We investigate whether a semanticaware framework can generalise from overt spoken commands to covert inner speech using scalp electroencephalography (EEG). Ten healthy participants produced four everyday commands (water, toilet, light, pain) in overt and covert phases. Overt trials included synchronized EEG, bilateral electromyography (EMG), and audio, whereas covert trials included EEG and EMG without audio. The proposed model learns a multimodal latent representation from overt data using supervised contrastive learning, ArcFace-based classification, and semantic alignment to sentenceembedding prototypes. Evaluation is performed in a subjectheld-out covert protocol with target-subject overt calibration: covert trials from the held-out participant are never used for training or model selection. Across subjects, the model achieves a mean overt-validation accuracy of 0.54 and a mean covert-test accuracy of 0.42 on the four-class task, above the 0.25 chance level. Covert predictions are moderately calibrated, and latentspace and semantic-retrieval analyses indicate that the learned representation preserves class structure and aligns with textbased semantic prototypes. These results show that multimodal overt supervision and semantic regularisation can support noninvasive decoding of everyday inner-speech commands, while highlighting subject variability and calibration as key challenges for future BCIs.