A conceptual framework for computational reproductions: Formal definitions and epistemic functions

Reproductions, which we define as “redoing activities” that work with the same collected data (or simulation source code) and aim to keep the computation as similar as possible, enable the verification of a scientific work. We seek to advance the understanding and conduct of reproductions by providing a conceptual framework and guidance to reproducers. In the first part, the epistemic function of reproductions and replications regarding mistakes – and faults as a consequence – is discussed. In the second part, the logic underlying reproductions is formulated, accompanied by formal definitions of central terms. The third and final part will provide practical guidance on conducting and reporting reproductions. In short, a reproduction involves performing another computation and comparing the results for consistency, that is, exploring the impact of computational choices on results before suspecting faults as a last resort. Computational choices by reproducers are necessitated by obstacles such as underspecification and grounds for conflicting choices. Regarding faults, four cases are emphasized: (1) incoherent descriptions, (2) different collected data (or simulation source code), (3) different data analysis source code, and (4) incorrect reporting of results. Separate from evaluating the consistency of results, reproductions can investigate their support for a claim. Finally, improper reporting practices can reduce the epistemic value of scientific works and their reproductions. Therefore, reproducers should also report the scope of their reproduction by detailing, among other things, which results were targeted, which computational stages were redone, and which descriptions were consulted.