Are we capturing individual differences? Evaluating the test-rest reliability of common experimental tasks used to measure social cognitive abilities

Background: Social cognitive skills are crucial for positive interpersonal relationships, health, and wellbeing and encompass both automatic and reflexive processes. To assess this myriad of skills, researchers have developed numerous experimental tasks that index automatic imitation, emotion recognition, empathy, perspective taking, and intergroup bias and have used these to reveal important individual differences in social cognition. However, the very reason that these tasks produce robust experimental effects – low between-participant variability – can make their use as correlational tools problematic. Aims: To perform an evaluation of test-retest reliability for common measures of social cognition. Method: One-hundred and fifty participants completed the race-Implicit Association Test (r-IAT), Stimulus-Response Compatibility (SRC) task, Emotional Go/No-Go (eGNG) task, Interpersonal Reactivity Index (IRI), Dot Perspective-Taking (DPT) task, State Affective Empathy (SAE) task, and indices of Explicit Bias (EB) across two sessions within 3 weeks. Results: Estimates of test-retest reliability varied considerably between tasks and their component measures: the IRI had good-to-excellent reliability (ICC = 0.76-0.83); EB (ICC = 0.70-0.77) and the eGNG had good reliability (ICC = 0.63-0.69); the SAE task had moderate-to-good reliability (ICC = 0.56-0.77); the r-IAT had moderate test-retest reliability (ICC = 0.49); the DPT had poor-to-good reliability (ICC = 0.24-0.60); and the SRC task had poor reliability (ICC = 0.09-0.29). Conclusions: Experimental tasks of social cognition are routinely used to assess individual differences but their suitability for this is rarely evaluated. Researchers investigating individual differences must assess the test-retest reliability of their measures.