Innovations in leadership measurement: Developing contextualized forced-choice metrics for collective leadership capacities

Traditional leadership assessments often rely on individualistic traits and self-reported rating scales, which are susceptible to response biases that weaken the measurement and quantification of the relational nature of complex psycho-social constructs such as leadership. This study details the development and initial validation of a forced-choice instrument designed to measure collective leadership capacities within the Teach For All network and contextualized to its education-centric framing. The instrument targets five dimensions of collective leadership in this context: Learn About Self, Embrace Systems Thinking, Belief In Students, Embodying Collective Mindset, and Beliefs about the Purpose of Education.To address the ipsative limitations inherent in traditional forced-choice formats, the study employed a Thurstonian Item Response Theory (T-IRT) model. The instrument was validated using a sample of 982 respondents, consisting of 382 network participants from Chile, Colombia, Mexico, and Peru, and 600 external respondents. Results indicated adequate model fit (RMSEA = .053) and significant factor loadings, providing evidence of structural validity. While internal dimensions were positively correlated, weak correlations with external leadership measures suggest the instrument is capturing a unique set of leadership capacities (specifically collective and educational in nature) that are not redundant with general social leadership metrics and thus differentiating what the instrument is capturing from broader social leadership or purpose-orientation metrics. Person-level scores also showed no evidence of the ceiling effects or negative skew often seen in Likert-based self-reports, suggesting the format successfully forced respondents to differentiate between equally desirable values and implying a more realistic profile of their leadership priorities.