Agree to Disagree? Human and LLM coder bias for constructs of marginalization

Human and LLM-based annotation of constructs of marginalization such as discrimination or hate speech comes with challenges. One such challenge is coder bias, the systematic variation in annotations driven by individual characterstics. To systematically understand the presence and sources of such biases, we conducted two studies and compared both. In Study 1, we surveyed crowdworkers on marginalization-related characteristics and analyzed their annotations of racism. In Study 2, we assigned personas to LLMs and studied variations in the annotations. Moreover, we compared human and LLM coder bias from a text-level perspective to account for textual properties driving variation. Our findings suggest that being affected by or aware of marginalization causes systematic variation in human annotations, while persona assignment significantly impacts LLM outputs. Against this backdrop, we advocate for ‘inclusive annotation’ which introduces variance by intentionally integrating coders with lived experience or awareness of the constructs, thereby enriching measurement quality.