Are we talking about the same psychopathic traits? Mapping semantic similarity across psychopathy scales using large language models

Self-report scales are the dominant method for assessing psychopathy in research, with numerous such instruments currently in existence. Whether these all measure the same underlying traits, however, remains unclear. In the present study, we systematically analyzed the semantic content of the most common psychopathy scales (N = 16) using state-of-the-art large language models and examined convergence and divergences at three levels of analysis. Our findings indicate that, at the scale level, most scales assess a largely common construct. However, there are notable exceptions, particularly the Dark Triad measures, which seem to capture only a limited subset of psychopathy. At the subscale level, we found evidence of jingle fallacies, whereby identical factor labels are used across measures despite substantial differences in item content. Finally, analyses at the item level revealed considerable heterogeneity in the traits being assessed. Specifically, we identified 23 distinct content clusters, yet most scales failed to cover all of them. This raises the question of whether current self-report scales adequately capture psychopathy in its entirety, or whether some scales assess traits that may not be central to psychopathy at all. We discuss the broader implications of these results for the fragmentation of psychopathy research and urge the field to work toward a shared consensus on the conceptualization and assessment of psychopathy.