Risk of bias in robustness reports

Bartoš et al. propose a new article format — Synchronous Robustness Reports (SRRs) — in which journals invite researchers to conduct alternative analyses of recently accepted studies. The goal is to assess the robustness of the original results by exploring how they vary across different analysis choices. However, the proposal does not distinguish between SRRs planned before versus after analysts have seen the data. This distinction is crucial because data-dependent analyses introduce a risk of bias from selective reporting. In this commentary, I argue that without safeguards, SRRs could inadvertently create more confusion than clarity about robustness.