Words Don’t Come Easy: How Lexical Difficulty of Items and Vocabulary of Subjects (Not) Affect Personality Assessment

Personality is commonly assessed using self-report scales comprising short personality-descriptive statements or adjectives. However, whether respondents understand these items receives little empirical attention. Across three studies, we investigated how vocabulary and lexical difficulty affect personality assessment. In Study 1 (n = 915), we compiled a broad pool of personality-descriptive adjectives and assessed their lexical difficulty via a multiple-choice vocabulary test. Based on this, we created adjective sets for each Big Five dimension with varying lexical difficulty. In Study 2 (n = 1106), a significant portion of respondents failed to choose the correct definition for harder adjectives. Personality ratings showed “hard” adjectives formed separate factors within each Big Five dimension. These factors showed low convergence with established personality items or easy adjectives, indicating they may reflect something other than the intended traits. Using locally weighted structural equation models we found no systematic variation in model parameters by subjects’ vocabulary, but vocabulary was modestly associated with the separate factor for hard adjectives. Study 3 (n = 542) examined whether the hard adjectives tapped different trait aspects (e.g., aspects of extraversion not covered by easier adjectives) or whether their shared variance reflected systematic response tendencies. Strong correlations among hard adjectives across Big Five dimensions suggested the latter: responses to hard adjectives likely reflect acquiescence rather than trait-specific variance. Overall, our findings highlight lexical difficulty as a key item characteristic that can introduce systematic bias. When participants lack the vocabulary to understand items, measures may capture erroneous interpretations, contaminating latent personality factors.