Reproducibility, replication, and preregistration

The replication crisis has raised our awareness that empirical findings may be much less robust than they seem; indeed, some highly influential psycholinguistic studies have seen failures to replicate. This shows that there are fundamental flaws in the way we (used to) go about doing empirical work. The current chapter discusses three methodological hallmarks that have emerged from the critical discourse surrounding the replication crisis: reproducibility, replication, and preregistration. It has become clear that they deserve to be regarded as integral elements of the scientific process, and the aim of this chapter is to help authors improve their work in the light of these insights. Following some initial terminological clarifications, we first turn to various aspects related to reproducibility, ranging from the FAIR principles for data sharing and data publication to computational tools and workflows ensuring the recoverability of statistical results. The focus then shifts to replication and replicability, with an outline of the continuum from direct to conceptual replication as well as advice on how to facilitate attempts by others to replicate our work. We also consider design, analysis and interpretation strategies that may be used to increase the replicability of our statistical conclusions and the linguistic claims they inspire. Finally, we discuss the essential role played by preregistration, where the researcher specifies their linguistic hypotheses and data analysis protocol prior to data acquisition; this form of academic self-discipline has culminated in the emergence of Registered Reports, a new publication format that is beginning to gain ground across the linguistic and cognitive sciences.