A Proof of Concept for the Measurement of Reliability as Conditional Determinacy

Read the full article See related articles

Discuss this preprint

Start a discussion What are Sciety discussions?

Listed in

This article is not in any list yet, why not save it to one of your lists.
Log in to save this article

Abstract

This paper presents a proof of concept for conceptualizing reliability as conditional determinacy. In psychometrics, reliability is typically defined within classical test theory andrelated variance-based frameworks as the proportion of observed variance attributable toa true or construct-related component. Although this formulation remains standard, itis limited in that it summarizes measurement quality at the level of aggregate varianceand therefore remains insensitive to how precision is distributed across items or whetherdistinctions in the underlying construct are preserved in observed responses.To address these limitations, the paper proposes an alternative perspective in whichreliability is defined at the level of the conditional distribution P(X | G), capturing theextent to which a measurement becomes determinate once the construct is fixed. Thisperspective leads to an entropy-based coefficient, denoted Γ, which quantifies residual uncertainty in observed outcomes conditional on the construct. Both a manifest formulationand latent approximations of this framework are outlined, including internal estimatorsbased on the information contained in other items.The proposed estimators are examined in a set of illustrative simulation scenarios andcompared with classical coefficients such as Cronbach’s α and McDonald’s ω. The resultssuggest that the proposed approach captures structural aspects of measurement, includinguneven item quality, distributional differences, and multidimensional organization, thatremain largely undetected by variance-based measures. The paper is intended as a preliminary proof of concept rather than a finalized framework. Its purpose is to make theproposed perspective explicit, demonstrate its feasibility in simple settings, and providea basis for further theoretical development, formal analysis, and empirical validation

Article activity feed