Behavioral and ERP Markers of Emotion Recognition: Comparing Embodied, Facial, and Emoji Stimuli

While extensive research has been conducted on emotion recognition from facial stimuli, it remains unclear how embodied emotional cues conveyed through body postures, gestures and actions (e.g., stick figures) compare with facial (human faces) and face-like symbolic representations (emoji faces) in shaping behavioral and neural responses. We employed a multimodal approach, combining behavioral measures (accuracy and reaction time) with electroencephalography (EEG) to examine the electrophysiological correlates of emotion recognition. Overall, sixty-six (equal number of males and females; 18–30 year old) participants identified positive, negative, and neutral emotions depicted in the three formats. Behavioral results showed significant effects of both emotion and format on accuracy and reaction time. Emoji faces were recognized with the highest accuracy and fastest reaction times, followed by stick figures and then human faces (p < .001, some comparisons p < .05). Thirty-four participants (17 males and 17 females) underwent EEG, which revealed distinct patterns of event-related potentials (ERPs). The Early Posterior Negativity (EPN) amplitude showed a significant overall effect of format. Post-hoc comparisons indicated that stick figures elicited greater (more negative) EPN amplitudes than human faces during positive emotion recognition (p = .003), and greater amplitudes than both emoji faces (p = .027) and human faces (p = .007) during negative emotion recognition. No significant differences were observed between emoji and human faces. N170 and Late Positive Potential (LPP) amplitudes did not reveal significant differences (all p > .05). Correlation analyses revealed no significant associations between ERP and behavioral measures. Thus, abstract, minimalistic representations like stick figures elicit enhanced early emotion-related processing despite similar early and later processing across formats.