Estimating the Prevalence of Unwarranted Disparities in Sentencing: Distinguishing between Good and Bad Controls

To minimise confounding bias and facilitate the identification of unwarranted disparities, sentencing researchers have traditionally sought to control for as many legal factors as possible. Over the past decade a growing number of researchers have questioned such approach, pointing at multiple legal factors are themselves subject to judicial discretion, so controlling for them leads to post-treatment bias. Here we use DAGs to provide a more formal and comprehensive assessment of the different types of bias that could be expected under different choice of controls. In addition, we put forward a new modelling framework to: i) facilitate the choice of controls under different definitions of sentencing disparities, and ii) reflect the model uncertainty stemming from the trade-off between confounding and post-treatment bias. We apply this framework to the estimation of race disparities in the US federal courts and gender disparities in the England and Wales magistrates' court. We find substantial model uncertainty for gender disparities and for race disparities affecting Hispanic offenders, rendering estimates of the latter inconclusive. Disparities against black offenders are more consistent, although, they are not strong enough to be seen as definitive evidence of racial discrimination.