Structural Equation Modeling Techniques for Estimating Score Dependability within Generalizability Theory-Based Univariate, Multivariate, and Bifactor Designs

Generalizability theory (GT) provides an all-encompassing framework for estimating accuracy of scores and effects of multiple sources of measurement error when using measures intended for either norm or criterion referencing purposes. Structural equation models (SEMs) can reproduce results obtained from GT-based analysis-of-variance (ANOVA) procedures while further extending those procedures to correct for scale coarseness, derive Monte Carlo based confidence intervals for key parameters, separate universe score variance into general and group factor effects, and determine subscale score viability. We demonstrate how to apply these techniques in R to univariate, multivariate, and bifactor designs using a novel indicator-mean approach to estimating absolute error. When representing responses to items from the Music Self-Perception Inventory (MUSPI-S) using two-, four-, and eight-point response metrics over two occasions, SEMs accurately reproduced results from the ANOVA-based mGENOVA package for univariate and multivariate designs and yielded score accuracy and subscale viability indices within bifactor designs comparable to those from corresponding multivariate designs. Corrections for scale coarseness improved score accuracy on all response metrics but to a greater extent with dichotomously scored items. Despite the dominance of general-factor effects, subscale viability was supported in all instances with transient measurement error leading to the greatest reductions in score accuracy.