The effect of multimodal input on L2 learners’ reading comprehension: A pre-registered, eye-tracking study

Multimodal materials (e.g., written text supplemented by images and/or audio)are commonplace in language classrooms. While they have been consistentlyshown to be beneficial for vocabulary acquisition, the efficacy of multimodalinput in scaffolding text comprehension is less clear. Conflicting findings havealso been reported in terms of the relationship between comprehension andattention to pictures as measured by eye tracking. In this pre-registered study,we provide further empirical evidence to this research base by testing a newpopulation, adult beginners, and by including a reading-only, no-imagecondition as the baseline. In a counterbalanced within-subject design, 65 learnersof Spanish were exposed to different parts of a story on an eye tracker underthree experimental conditions: Reading Only (RO), Reading + Image (RI), andReading + Image + Audio (RIA). Results revealed that comprehension washigher in the RIA and RI than in the RO condition, indicative of the usefulnessof the audio and/or the image. The number of looks at the images was higher inthe RIA condition, confirming that the audio allowed readers to attend to thepictorial information provided. Attention to the image, however, positivelypredicted comprehension scores only for the RIA condition.