Robust Standard Errors and Confidence Intervals for Standardized Mean Difference

The standardized mean difference (SMD) is a widely used effect size metric for assessing intervention effects in experimental designs and for quantifying disparities in observational studies. Its central role in power analysis and meta-analysis demands robust standard errors (SEs) and confidence intervals (CIs) that remain reliable under unequal variances and in the presence of data contamination. To address this, I proposed heteroscedasticity-consistent (HC) estimators for the variance component of SMD and enhanced their robustness using trimmed means and Winsorized variances (TW estimators). Twenty SE and CI estimators were evaluated through extensive Monte Carlo simulations, varying effect sizes, sample sizes, variance ratios, sample size ratios, and distributional shapes. Performance was assessed using coverage, Type I error, relative bias in SE and power, and CI overlap. Additional comparisons included alternative CI construction methods (normal approximation, central t, lambda-prime distributions) and bootstrap approaches (percentile and bias-corrected accelerated). Results showed that CIs based on the lambda-prime distribution achieved the best coverage, lowest bias in power, and greatest overlap with simulation-based CIs. Performance of HC estimators depended on the sample size and variance ratios. Generally, SMD with HC2 (Welch-Satterthwaite), HC3 (Efron’s Jackknife), HC5, and their TW versions provided reliable inference under unequal variance and contamination. TW estimators were most useful under symmetric contamination but offered limited benefits for skewed or mean–variance linked distributions. Bootstrap methods combined with TW estimators worked remarkably well under all conditions. Finally, implications for inference, power, and meta-analysis were discussed.