Item-Level Heterogeneous Treatment Effects in Instrumental Variables Regression

Recent research has highlighted that modeling heterogeneous treatment effects across the individual items on an outcome measure can yield additional insights into interventions’ effects beyond what is available from comparisons of average scores. So far, this work has generally focused on randomized controlled trials. However, in social science research, randomization often does not hold perfectly, or may be impossible or impractical. This can lead to treatment endogeneity—correlation between the predictor and the error term of the outcome. Instrumental variables regression (IVR) is a popular method for correcting the resulting bias by introducing an exogenous variable that drives change in the predictor but is otherwise uncorrelated with the outcome. Though typically estimated via two-stage least squares, IVR can also be estimated in a latent variable framework. We develop an explanatory item response modeling approach for estimating the average of treatment effects that are heterogeneous across the items used to measure a latent outcome in an IVR, supporting valid causal inferences about impacts of nonrandom treatments/dosages on test items. We illustrate the use of the approach via an empirical example analyzing item-level variability in sensitivity to time spent on homework using data from a large-scale international assessment of advanced high school mathematics.