Confidence Intervals for the Difference Between Proportions after Multiple Imputation: A simulation study comparing different strategies

Background The difference between proportions and their confidence intervals are important statistical methods to determine group differences in randomized trials and observational studies. In these studies missing data may severely bias results and should be handled carefully. The Newcombe-Wilson, Wald and Agresti-Caffo methods are commonly used to calculate confidence intervals between proportions. It is unclear how these methods behave in combination with multiple imputation to handle missing data. Methods A simulation study was conducted to compare the coverage probability and interval lengths of the Newcombe-Wilson, Wald and Agresti-Caffo methods to calculate the confidence intervals for the difference between proportions after multiple imputation. Evaluated was the performance in MCAR and MAR missing data of 10% and 30%, using large and small differences between proportions and small and large sample sizes. Results In all simulation scenarios the methods under study performed equally well and were close to the desired coverage probability of 0.95, with similar interval lengths. Conclusions It can be concluded that simple methods as the Wald and Agresti-Caffo confidence intervals perform similar as a more complex procedure as the Newcombe-Wilson intervals. The Wald intervals showed undercoverage which makes the Agresti-Caffo confidence intervals a strong alternative for the more complex Newcombe-Wilson intervals method.