Information-Based Sequential Monitoring in Linear Models

Researchers in psychology often use statistical inference through effect size estimation and hypothesis testing. Traditionally, the required sample size for a study is determined \textit{a priori} as a combination of statistical design choices and assumed characteristics of the population. Because these population parameters are unknown, fixed-sample designs rely on assumptions that are often difficult to justify and can lead to underpowered or inefficient studies. In this paper, we discuss as an alternative a sequential approach that monitors the total Fisher information for the effect of interest, but not the estimated effect itself. Depending on the goal of the study, a target level of total Fisher information is prespecified that corresponds to obtaining either a full confidence interval width around the effect of interest with a desired width, or to a hypothesis test that achieves the target power to detect a minimally meaningful effect. We focus specifically on the setting of linear regression models with normally distributed errors. Unlike fixed-sample designs, the sequential monitoring approach does not require prior knowledge of population parameters, but the price to pay is that the final sample size is unknown during the study. We therefore propose a nonparametric bootstrapping approach to construct prediction intervals for the final sample size of the study. These intervals are based on accumulated data, shrink as the stopping rule nears and provide researchers a transparent means to potentially modify the sampling regime or the statistical design choices of the study.