When our measurements are different every day: an ML-SEM simulation study on within-person nonuniform measurement bias in intensive longitudinal data

This simulation study evaluated how model fit in multilevel structural equation models (ML-SEM) is affected by within-person nonuniform measurement bias in intensive longitudinal data (ILD). This kind of bias would be given if item discrimination (i.e., their factor loadings) in multiple-item questionnaires varied within person across time. Prior simulation studies and ILD studies tend to assume no such within-person bias, while such a bias implies that relations within measurement points are not comparable across time. We simulating ILD under 450 conditions with various sample sizes, retesting frequencies, ICCs, and introduced within-person nonuniform measurement bias. We then investigated model (mis)fit in ML-SEM. Type I error was well below nominal level. The c² statistic and CFI outperformed the other fit indices (RMSEA, SRMR-w, SRMR-b), with the effects being conditional on all design factors. While ecological momentary assessment motivated our study, our findings are applicable to other research settings that yield data with the same generally hierarchical structure (e.g., ambulatory assessment, daily diary studies, experience sampling methods). We conclude with practical recommendations on samples sizes and re-testing frequencies to ensure guarding against within-person nonuniform measurement bias.