Beyond Average Scores: Identification of Consistent and Inconsistent Academic Achievement in Grouping Units

Research on academic performance typically revolves around average achievement scoresof students or schools. Focusing solely on averages can miss important aspects of thelearning experience. The recent development of mixed effects location scale models(MELSM) has provided a modeling technique that incorporates a scale model thatcaptures and explains the (in-)consistency of academic achievement within the cluster ofinterest. Here, we formally introduce an extension to the MELSM, a Spike and SlabMixed-Effects Location Scale Model (SS-MELSM), for simultaneously modeling locationand scale parameters while incorporating a spike and slab prior to select or shrink randomeffects. Our approach involves identifying clusters with unusually large or smallwithin-cluster variance in academic achievement, which can indicate overly inconsistent orconsistent academic achievement. We apply the SS-MELSM to a dataset of 160 schoolsfrom the Brazilian Evaluation System of Elementary Education (Saeb) to illustrate its usein educational data analysis. Moreover, we show how to compare models with varyingparameters regarding expected predictive accuracy. The results demonstrate that theSS-MELSM successfully identifies schools with unusually high and low consistency inmathematics achievement and that school- and student-level SES were relevant covariateswhen modeling the location and scale components. The methods presented in this paperare implemented in the R package ivd.