(In)validity by Design? Separating Measurement Validation from Substantive Research in HCI

This position paper argues that the verity of empirical findings in HCI hinges on rigorous measurement practices of latent psychological constructs — a foundation too often undermined by ad-hoc and unvalidated scale development and modifications. We contend that current practices lead to distorted, only seemingly evidence-based inferences while squandering valuable resources. To mitigate these issues, we propose a 'validity by design' approach that separates measurement validation from substantive hypothesis evaluation. Central to our proposal is the establishment of a collaborative consortium tasked with defining core constructs, systematically validating standardized instruments, and disseminating them through open-access platforms. This coordinated effort — augmented by the active involvement of key thought leaders and targeted funding — will empower the HCI community to foster proactive measurement validation, thereby enhancing both scientific rigor and practical impact.