Eye movements reveal a dissociation between prediction and structural processing in language comprehension

Reading seems smooth and effortless, but this appearance is deceiving: In the process of extracting a meaning from a text, our eyes linger much more on some words than others, and we often reread earlier portions of the text. These disruptions are particularly common in syntactically challenging sentences. What explains this complex pattern of eye movements? One prominent hypothesis explains these patterns as a special case of the impact of a word's predictability (its surprisal) on the difficulty of recognizing the word. An alternative hypothesis attributes these disruptions to syntactic structure-building operations that apply after word recognition. Earlier attempts to address this debate have been inconclusive because of small numbers of participants, coarse measurements that are ill-suited to disentangling these competing views, and a limited range of predictability estimates. Here, we conduct a large-scale study (n = 368) examining eye movements during the reading of syntactically challenging sentences, using 407 types of predictability estimates from large language models with multiple architectures and training settings. We find a stark dissociation: Early reading measures are well approximated by language model surprisal, but syntactic disambiguation incurs a significant additional cost, reflected in an increase in rereading that is not explained by surprisal. We further show that when rereading parts of the sentence, readers strategically target the words most useful to amend the structure of the sentence. We conclude that linguistic knowledge guides moment-by-moment reading in two dissociable ways: Forward reading is driven by the word's surprisal, and backward reading reflects syntactic structure building operations.