A<i> p</i>-Value Paradox in Proportion Tests and Its Resolution

This technical note investigates a p-value paradox that emerges in the conventional proportion test. The paradox isdefined as the phenomenon where &ldquo;decisions made on the same effect size from data of different sample sizes may be inconsistent.&rdquo; It is illustrated with two examples from clinical trial research. We argue that this p-value paradox stems from the use (or misuse) of p-values to compare two proportions and make decisions. We propose replacing the conventional proportion test and its p-value with estimation statistics that include both the observed effect size and a reliability measure known as the signal content index (SCI).