Synthesizing Single-Case Experimental Designs: Modeling Complex Data Structures

The application of multilevel modeling to analyze data from multiple single-case studies can lead to challenges in selecting and estimating an appropriate model. Models that are too complex may not be well estimated and models that are too simple may lead to biases in effect estimates and their standard errors. Researchers aim to select a model that answers their research question(s) and is simple enough to be estimated well, but complex enough that effect estimates have little to no bias and inferences are accurate. This Monte Carlo simulation study indexed the consequences of using mixed linear models with varying degrees of complexity to approximate nonlinear processes that vary within and across cases. We found that modeling the treatment phase data with two linear pieces, produced less biased treatment effect estimates and more accurate inferences than modeling the treatment phase with a quadratic trajectory, and that estimating more complex heterogeneous variance structures led to more accurate confidence intervals than assuming a simpler variance structure.