Neural mechanisms of visual cross-modal conflicts in auditory working memory: an ERP study of semantic and non-semantic distractors

Visual cross-modal conflicts occur when visual distractors interfere with auditory task performance. However, the specific effects of semantic and non-semantic visual distractors, particularly their neural dynamics, remain poorly understood. This study investigated the impact of semantic and non-semantic visual cross-modal conflicts on auditory working memory performance and its neural mechanisms, as revealed by an electroencephalogram (EEG). Thirty-nine healthy, right-handed young adults participated, completing the paced auditory serial addition test (PASAT) under three conditions: no visual distractor, semantic visual distractors (digits), and non-semantic visual distractors (symbols). Both semantic and non-semantic distractors significantly reduced task accuracy compared to the no-distractor condition, with semantic distractors producing stronger interference, as reflected in longer reaction times (RT) and higher omission rates (MR). Event-related potential (ERP) analysis revealed that both conflicts elicited larger P200 amplitudes, more negative N200 amplitudes, and delayed P300 latencies. The semantic conflict elicited more negative N200 amplitudes at the frontal region, while the non-semantic conflict evoked smaller P300 amplitudes and longer P300 latencies at the central region. Correlation analysis revealed significant relationships between changes in behavioral performance and ERP components. Specifically, ∆P300 amplitude negatively correlated with ∆RT and ∆Error rate in the semantic conflict condition. ∆N200 amplitude negatively correlated with ∆Accuracy and ∆RT, and ∆P300 latency positively correlated with ∆MR in the non-semantic conflict condition. These findings demonstrate that visual cross-modal conflicts disrupt auditory working memory task performance, involving both specific and general mechanisms for resolving semantic and non-semantic conflicts.