Repeatability in Monte Carlo Simulation Studies

Monte Carlo simulation studies are a primary methodology used to evaluate statistical methods which then are used in psychology and other social-behavioral sciences. While the replication crisis in social-behavioral science led to deep reflection on the repeatability of research studies and findings, the same has not yet occurred for Monte Carlo simulation studies. Recent evidence suggests there are reasons for concern about the repeatability ofMonte Carlo simulations, which could have major implications for social-behavioral sciences more broadly, given the impact these studies have on analytical approaches used across fields. In this paper, we adopt a specific approach to describing and differentiating repeatability terms (replication, reproduction, and robustness) (Nosek et al., 2025) and combine this with a framework for describing the stages of Monte Carlo simulations (Sigal& Chalmers, 2016), to better describe repeatability and it’s implications for Monte Carlo simulations. This approach demonstrates the importance and distinct value of each type of repeatability, as well as identifies the role of open data and code, and other verification studies, like multiverse analyses and registered reports. Monte Carlo simulation studies are important for informing statistical practice, and ensuring the repeatability of these studiesis tantamount to the appropriate use of statistical methods in psychology and beyond.