Initial signs of learning: Decoding newly-learned vocabulary from neural patterns in novice sign language learners

Learning a new language requires the brain to map new perceptual cues onto real-world semantic concepts. Studies of semantic processing suggest that these representations are at least partially language-independent: despite differing low-level perceptual features, homologous words can evoke overlapping neural patterns associated with a shared underlying meaning. A few studies have leveraged this relationship to examine the emergence of knowledge in the brains of novice learners, but so far all have focused on learners of spoken languages. Evidence from bimodal (sign and speech) bilinguals suggests that neural representations of signs and spoken words may be language-dependent on the individual item level but share representations of broad semantic features. However this remains to be tested in novice signers, who may rely more on their established spoken language representations. We present two experiments in which hearing English speakers underwent brief training in American Sign Language (ASL). In Study 1, participants were all trained to ceiling on a set of signs, while in Study 2 the task difficulty was increased to produce individual differences in performance. . Using representational similarity analysis (RSA) with fMRI, we identify brain regions where neural patterns reflect semantic relationships between nouns in both languages. . Moreover, in Study 2 participants, we show that multivariate neural measures of semantic representation of ASL in several frontal, temporal, and occipital regions reflect individual differences in comprehension. These results suggest that in contrast to prior work in bilinguals, novice signers show semantic representation at the item level in partially overlapping regions, and the degree of multivariate correlation can track individual-level shifts in understanding in the earliest stages of sign language learning.