Replicative significance index (RSI): A simulation-based metric for statistical inference and reproducibility

Background: Conventional null hypothesis significance testing, which relies heavily on p-values and confidence intervals, remains central to scientific research. However, both measures are subject to variability and may yield fragile or misleading conclusions. This has contributed to reproducibility concerns and the need for more informative inferential tools. Methods: We propose the Replicative Significance Index (RSI), a simulation-based metric that estimates the proportion of hypothetical replications that achieve statistical significance under identical design assumptions. The RSI directly models the stability of observed significance, in contrast to post hoc p-value interpretation or traditional power analysis. We further introduce a tail-augmented RSI (RSI*), which incorporates the depth of significant p-values through a tail-dominance adjustment. RSI and RSI* were implemented in R and demonstrated via conceptual examples, simulations under varying precision, and illustrative case scenarios of fragile significance and hidden robustness. Results: The simulations revealed that the RSI discriminates between superficially similar p-values that differ markedly in replicability and highlights cases where confidence intervals and p-values may overstate or understate robustness. RSI* further rewards distributions concentrated in the lower tail, distinguishing robustly reproducible findings from borderline significance. An R package and interactive Shiny web application are provided to facilitate implementation.