Bridging null hypothesis testing and estimation: A practical guide to statistical conclusion drawing from research in psychology

A well-known problem of null hypothesis significance testing is that it cannot be used to find support for the null hypothesis. A common solution for this is to replace the exact 0 value by an interval associated with values that are close to zero. This approach is denoted as equivalence testing and is a special case of procedures that test intervals of values against each other. Smiley et al. (2023) recently published a unified framework of statistical inference and suggested a straightforward method of testing all sorts of interval-based hypotheses in a unified way. The present paper discusses three alternative general approaches, based on Bayesian analysis, which have the advantage that the ensuing probabilities can be interpreted as probabilities of the population parameters, rather than probabilities of the data (as is the case with frequentist methods). These methods (in some form) have been previously suggested, but here we bring them together and show how they can be used for Smiley et al.’s full unified framework of statistical inference, now complementing it with three Bayesian counterparts. In particular, it is shown how each of the methods work in the analysis of a leading example data set involving a test on proportions. Subsequently, their relative pros and cons are discussed, and it is explained how the methods can be used for many statistical analysis questions in practice, using R and/or JASP. This is illustrated on an empirical data set for comparing means of two groups.