Entropy’s dilemma: Facilitatory and Inhibitory Effects of Information Entropy during Language Processing

Processing words during language comprehension is facilitated when they are more predictable in the context they occur. However, contexts usually lead to prediction of multiple rather than a single word. We examined how the distribution of multiple plausible words following a context, i.e., conditional entropy, influences lexical processing. Greater conditional entropy simultaneously corresponds to activation of more semantic features, which should facilitate processing, and activation of more lexical competitors, which should inhibit processing. Participants (N=58) completed two experimental sessions 14-21 days apart. In session 1, they produced up to 8 completions for sentence fragments (N=648) missing a final, target word (e.g., banana) along with probability values for each response. In session 2, they read the same sentences including the target word while their eye-movements were tracked. We computed conditional entropy at the trial level (i.e., for each participant, for each sentence). To tease apart the semantic feature activation and lexical competition components of conditional entropy, we calculated semantic entropy; the sum of semantic similarity values between target words and the responses produced during sentence completion (e.g., mango, orange, coconut etc.). The remainder of conditional entropy would then capture lexical entropy corresponding to lexical competition. The results revealed that semantic entropy facilitated lexical processing, but lexical entropy inhibited it. A re-analysis of an independent self-paced reading dataset (N participants=111, N items=647) revealed the same pattern. These results suggest that conditional entropy should be decomposed to semantic activation and lexical competition, corresponding to facilitation and inhibition during language processing, respectively.