What Makes Co-Speech Gestures Memorable?

Adults consistently remember, and forget, certain visual stimuli––particular static images or dance movements––regardless of their personal experience with those stimuli. Here, we ask whether this phenomenon, dubbed the memorability effect, extends to a different type of visual stimulus––co-speech gestures, which are hand movements speakers spontaneously produce when they talk. Investigating the kinds of gestures that are memorable and how the memorability effect interacts between modalities is important for understanding how gesture can be incorporated into instruction to promote student learning. We ask, first, whether there is a memorability effect for gesture and, if so, whether semantic meaning features and/or visual form features are responsible for this effect. We created 360 10-second audiovisual stimuli by video recording 20 actors producing unscripted natural speech and gestures as they pretended to explain Piagetian liquid conservation to a child. We then tested online participants’ memories using a study-test paradigm for video, audio, and audiovisual versions of the stimuli. Participants showed memory consistencies in all three conditions, and memorability for gesture+speech (audiovisual stimuli) was predicted by the memorability of both its gesture and speech components. We then quantified features of the gestural stimuli using 3 methods: trained coders, automatic computational analysis, and online crowd-sourcing. Gestures are more memorable when they are more informative, and are most memorable when originally produced with memorable speech. These findings provide insight into the impact that multi-modal communication has on memory, and offer a basis for investigating whether increasing the memorability of instruction can promote student understanding.