Evaluating biomarkers and prediction models with E2P Simulator

Precision psychiatry aims to identify biomarkers and develop prediction models to improve diagnosis, prognosis, and treatment selection for mental illness. Despite extensive research, translating findings into clinically useful tools remains challenging. Here we argue that one key contributor to this translational gap is a pervasive misalignment between routine statistical analytic practices and the criteria for clinical utility. To address this, we introduce E2P (effect-to-prediction) Simulator, an interactive web tool for estimating the real-world predictive value and clinical utility of biomarkers and prediction models by accounting for real-world outcome base rates and measurement reliability – a procedure we term predictive utility analysis. Similar to how power analysis helps optimize research for statistical significance, predictive utility analysis can help optimize it for practical significance. The interactive nature of E2P Simulator makes this approach accessible to non-statistician researchers while also providing publication-ready figures to aid with transparency and standardization of reporting. We demonstrate its application in three key areas: diagnostic (depression, Alzheimer’s disease), treatment response (antidepressants), and risk (suicide attempts and psychosis onset) prediction, highlighting conditions under which we may expect clinically meaningful advances. While we focus on translational challenges in psychiatry, the framework and tools presented here address general statistical challenges and are broadly applicable across biomedical and behavioral sciences.