Information-making processes in the speaker’s brain drive human conversations

The neural basis of spontaneous speech production, in which speakers efficiently and effortlessly generate utterances on the fly to express their thoughts, is among the least understood aspects of human cognition. This study utilizes information theory, contemporary Large Language Models (LLMs), and approximately 100 hours of high-quality spatiotemporal ECoG recordings of speakers engaged in spontaneous conversations to explore how the speaker’s brain conveys information during everyday interactions. Information theory defines information as the reduction of uncertainty (Shannon entropy). It lays the theoretical foundations for why listeners actively predict upcoming words (information-seeking) before a word is spoken while enhancing the processing of unexpected, information-rich words after they are perceived. But what happens when speakers generate (information-making) these improbable, information-rich words in the first place? We analyzed continuous electrocorticography (ECoG) recordings collected during hours of real-life, 24/7 conversations to address this question. Using LLMs (Llama-2 and GPT-2), we estimated the probability of each word based on its context, categorizing them as either improbable, information-rich words or predictable, information-thin words. We then extracted word-based non-contextual embeddings from these models and employed neural encoding techniques to examine brain activity during speech production and comprehension. Our findings reveal a striking contrast in how the brain handles improbable, information-rich words while speaking versus listening. During speech comprehension, we identified two distinct neural phases: one preceding word onset, associated with predictive (information-seeking) processing, and another following word onset, linked to enhanced information processing of unexpected words. Conversely, in the speaker’s brain, we found, for the first time, enhanced pre-word-onset encoding for improbable, information-rich words versus probable words. The results remained strong and clear even when we narrowed down the analysis to a shared set of words that were unlikely in one context and likely in another. Since information-rich words are statistically unpredictable, this suggests the speaker’s brain aims to produce linguistic output that defies listeners’ expectations. However, we also point out that predictability alone is insufficient to generate meaningful words, highlighting a gap in information theory and LLMs that neglects how speakers intentionally choose information-rich words to convey novel meanings.