Initial signs of learning: Decoding newly-learned vocabulary from neural patterns in novice sign language learners

How do novice language learners represent semantic information in their new language? The extent to which multiple languages are supported by divergent or overlapping semantic representations in bilinguals has been well-studied, but less is known about how new knowledge is integrated into established representational networks at the earliest stages of acquisition. Furthermore, examining language across modality (sign vs. speech) can provide unique insight into language unconfounded by perceptual features. We present two experiments in which hearing non-signers underwent brief training in American Sign Language (ASL) followed by fMRI scanning. Across both datasets (N=50), we use representational similarity analysis (RSA) to identify brain regions where neural patterns reflect semantic relationships between stimuli. In Study 2 (N=40) we show that multivariate neural measures of semantic representation in several frontal, temporal, and occipital regions reflect individual participant-level comprehension. These results demonstrate the role of frontal and temporal regions, especially bilateral superior temporal sulcus, in representing semantic content across language and modality in novice learners.