A novel approach to map the causal impact of brain stimulation on semantic processing with language models

Non-invasive brain stimulation (NIBS) studies on semantic cognition hold the promise of revealing the functional relevance of brain areas through causal intervention. A primary challenge, however, is that findings are often interpreted through binary distinctions between sets of stimuli (e.g. related/unrelated words, same/different semantic category). This approach ignores the analysis of individual words, which mirrors every-day language use and is crucial for understanding semantic cognition. In this work, we used semantic similarity, as measured by a language model, to investigate how Transcranial Magnetic Stimulation (TMS) effects on semantic cognition unfold at the level of individual words. We re-analyzed 5 publicly available TMS datasets, covering multiple stimulation sites and lexical semantics tasks. We propose a simple methodology that can straightforwardly be applied to any TMS experiment on semantic cognition, and showcase its potential to generate new insights. We modelled trial-level response times using the language model and computed the correlation between the two. We also repeated the analyses for two lower-level variables (word frequency and length). Importantly, for each dataset, we compared correlations for effective and control (sham or vertex) stimulation conditions. We found that, for the language model, correlation was almost always significantly different depending on the type of stimulation (effective or control). Our results provide evidence that the stimulation effect interacts with the meaning of individual words. However, a similar pattern emerged in some cases for word frequency and length, suggesting that the effects of TMS on cognition can be widespread, well beyond their intended functional target. Collectively, our results demonstrate that language models provide new insight into the impact of neurostimulation on semantic processing, complementing standard measures.