Charting the pitfalls of disproportionality analysis

Background: Disproportionality analysis is used by many pharmacovigilance organizations for detecting and assessing signals of potential adverse drug reactions. However, its goal is often misunderstood and the approach misapplied, leading to erroneous conclusions due to neglected violated assumptions.Objective: To illustrate how the simplistic use and interpretation of disproportionality analysis can lead to incorrect conclusions. Methods: Using VigiBase, the WHO global database of adverse event reports, and the Information Component disproportionality metric, we provided selected examples to highlight common sources of error that can introduce spurious associations or lead to missing important signals.Results: We illustrate the following types of pitfalls: confounding (by age, sex, indication, comedication), effect modification (by age), notoriety bias, masking, misclassification (by miscoding, incomplete or imprecise event retrieval), neglecting report utility, and violated independence assumption. Additionally, we show how sophisticated analyses may introduce new bias or amplify existing ones, such as collider bias or masking amplification. Conclusion: Due to its pitfalls, disproportionality analysis plays a supportive rather than decisive role in signal detection and assessment. Careful design and interpretation of disproportionality analysis, with appropriate subgrouping and clinical assessment, are essential. While subgrouping can mitigate some pitfalls, it reduces sample size and may introduce or amplify existing biases. Further development of tools to detect and mitigate biases and assess the risk of bias in published disproportionality analyses is needed.