Neural tracking of speech increases in the presence of representational gestures during face-to-face dialogue

Representational gestures have been shown to be beneficial for speech comprehension especially in adverse listening conditions, as they provide meaningful information related to speech. However, it is still unclear whether seeing representational gestures directs additional attention towards the speech signal. In the current study, we addressed this research question by investigating neural tracking of speech in the presence of representational gestures during unscripted dialogue with and without background noise. Dyads performed a tangram-based referential communication task, while audio-visual behavior and dual-EEG were recorded. Representational gestures increased neural tracking of speech in both delta and theta frequencies and in both dialogue conditions in comparison to control movements. This finding suggests that seeing representational gestures directs attention to the processing of both acoustic and linguistic information of the speech signal. As no differences in neural tracking of speech were observed between dialogue conditions, we believe that dyads made use of different strategies to overcome the noise interference. In sum, we argue that neural tracking of speech represents a flexible mechanism which can be modulated by attentional processes to meet different cognitive and communicative demands during face-to-face interaction.