Doubly Protective or Doubly Fragile? A Comparison of Doubly Robust Approaches for Estimating Average Treatment Effects

In nonexperimental studies, obtaining an unbiased estimate of the average treatment effect (ATE) typically requires two key assumptions: that all relevant covariates are measured (i.e., no unmeasured confounding) and that the statistical model used for covariate adjustment is correctly specified. Two common approaches for adjustment are outcome regression and propensity score weighting. To mitigate bias from model misspecification, doubly robust methods combine both approaches, ensuring unbiased ATE estimates if either the outcome model or the propensity score model is correctly specified. In this study, we review four doubly robust methods that have received considerable attention in the methodological literature but remain underutilized in psychological research: augmented inverse probability weighting, regression weighted by the inverse propensity score, regression incorporating the inverse propensity score as a covariate, and calibrated propensity score weights. Using two simulation studies, we compare these methods with traditional regression and inverse probability weighting estimators. Our results suggest that doubly robust methods—particularly regression weighted by the inverse propensity score—offer greater protection against bias from model misspecification across various data-generating scenarios. We also discuss practical considerations for implementing doubly robust methods, including weight normalization, propensity score truncation, and potential efficiency losses due to overfitting. The different methods for estimating the ATE are illustrated in a data example.