Robust tests should be the default, not the backup
Discuss this preprint
Start a discussion What are Sciety discussions?Listed in
This article is not in any list yet, why not save it to one of your lists.Abstract
This opinion piece summarizes the epistemic benefits of using robust statistical tests in the falsificationist tradition over standard tests such as the t-test, ANOVA, and conventional tests in ordinary least squares regression. Robust alternatives like robust linear regression do not hinge on assumptions like normally distributed errors with equal variances or the inconsequentiality of extreme values and outliers. Tests with these broad robustness features act agains nonreplication that occurs solely because data anomalies arise differently across studies. Using them from the outset sidesteps the pitfalls of making a data-based decision about whether a standard test is applicable. The common practice of conducting them in addition, commonly in response to data inspection, yields multiple test results. I argue that these should be avoided when a binary decision must be reached, for example, conducting further research on the basis on the assumption that an effect exists. Practically, using a single test simplifies analysis. While R offers numerous robust methods, the ones that provide broad robustness are largely restricted to linear models.