Bayesian Treed Regressions for Estimating Heterogeneous Trajectories of Test Scores in Large-scale Educational Assessments

In large-scale educational assessment, students’ academic outcomes can evolve over time, due to various effect modifiers. For instance, parents’ education level, a common modifier, has been found to have time-varying effects on children’s test scores, leading to heterogeneous academic trajectories. To model these varying trajectories, parametric models like hierarchical linear models (HLMs) are commonly deployed. While these highly parametric methods are interpretable, their predictive capacities have not been rigorously compared to those of more flexible nonparametric approaches. We present the results of systematic simulation studies and empirical analyses comparing traditional parametric models to Bayesian nonparametric models based on Bayesian Additive Regression Trees (BART). Our findings indicate that BART-based alternatives can (1) achieve comparable out-of-sample prediction as HLMs, after accounting for individual variability over time; (2) provide a reasonable amount of interpretability and uncertainty quantification; and (3) do not incur much more computational burden. This work contributes to the existing social science literature by demonstrating that Bayesian treed regression methods are viable and feasible alternatives to conventional HLMs for predicting heterogeneous trajectories across time.