Variance Estimation for Assessing Healthcare Providers’ Performance using log Standardized Incidence Ratio
Listed in
This article is not in any list yet, why not save it to one of your lists.Abstract
Introduction
In healthcare providers’ performance assessment, standardized incidence ratios (SIRs) are essential tools used to assess whether observed event rates deviate from expected values. Accurate estimation of variance in these ratios is crucial as it affects decision-making regarding providers’ performance. There is little data on how the choice of these variance estimation methods affects decision-making. In this paper, we compared three methods, namely, delta-method, bootstrapping and Bayesian approaches, to estimate the variance of the logarithm of SIR (Log-SIR).
Methods and analysis
Using patient-level data from Australia and New Zealand Dialysis and Transplant Registry (ANZDATA) for 2005-2023, we used a random effects model to predict treatment at home one year after starting treatment. We compared the three approaches (with over 5000 iterations for bootstrapping and MCMC sampling) using bias, variance and mean square errors (MSE) as performance measures. Using the three methods, funnel plots were used to compare the hospitals’ performance in treating Indigenous and non-Indigenous patients close to home, as a service-level measure of equity.
Results
The bias values across all methods are similar, with Bayesian narrowly having the lowest bias (0.01922), followed by the delta-method (0.01927) and Bootstrap (0.02567). In addition, the Bayesian exhibits the lowest variance (0.00005), indicating more stable and less dispersed estimates. The delta-method has a higher variance (0.00016), while Bootstrap has the highest variance (0.00027), meaning it introduces more uncertainty. Finally, the Bayesian has the lowest MSE (0.00042), indicating better overall accuracy while the Bootstrap has the highest MSE (0.00094), showing it is the least reliable method.
Conclusion
We demonstrate that these methods can be used to measure equity for patient-centred outcomes, both within and between service providers simultaneously. The choice of variance estimation method is critical and heavily affects the interpretation of the performance of health service providers. We favour the Bayesian MCMC method. The Bayesian MCMC method found to be better approach.
