Statistical Analysis for the Changing Criterion Design: Separating Intervention Effects from Secular Change

Interrupted time-series research designs have recently become the major alternative to randomized clinical trials in the health sciences. Methods used to analyze the simple two-phase (AB) version of this design are most frequently (1) some type of segmented time-series regression analysis or (2) an ARIMA time-series intervention model. Although the AB design is by far the most popular time-series intervention design in the health sciences, more complex design variants provide much stronger evidence for causal effects. One problem with these more complex designs is that statistical methods of analysis for them are not well developed. A new statistical analysis for the changing criterion design is described here. A potential weakness of the changing criterion design is that the baseline phase, which is assumed to be stable (i.e., non-trending over time), may contain a secular trend. If changing criterion interventions are introduced when a secular trend is present the interpretation of observed outcome change is ambiguous; secular trend and manipulated interventions are both viable explanations for apparent change. This is a situation in which a statistical method that can separate the possible intervention effects from the secular change may be desirable. The analysis proposed in this article is carried out in three stages. The first stage yields information regarding the overall monotonic relationship between the criterion for reinforcement (or treatment intensity) and outcome behavior. The second stage provides individual phase-change effect estimates that are independent of secular effects and additional descriptive and inferential measures of change. The third stage provides a test to evaluate the overall relative adequacy of the two competing models of behavior change. Ordinary least-squares regression (OLS) and two new double bootstrap routines (Zhang, McKean, & Huitema, 2022) are available for estimating the parameters of the relevant models. Key Words: Interrupted Time-Series Designs, Intervention Analysis, Multiple-Phase Time-Series Designs, Secular Effects Confounding.