Beyond Total Scores: Enhancing Psychotherapy Outcome Prediction with Item-Level Scores

Objective: This study aims at improving dropout and treatment non-response prevention by optimizing the performance of models for their prediction through the integration of item-level data. Method: Routine data from 1277 patients (mean age = 36.95, age standard deviation = 13.64; 64.77% female) treated at Osnabrück University was used to train and evaluate 20 machine-learning algorithms and five ensemble models. Measures included sociodemographic information, Outcome Questionnaire-30 (OQ-30), Questionnaire for the Evaluation of Psychotherapy (FEP–2), Emotional Well-Being Questionnaire (EMO), Symptom Checklist (SCL–90–R), and the Inventory of Interpersonal Problems (IIP-32). Prediction models were trained with nested cross-validation and validated in a holdout sample. SHAP values were extracted for the best resulting model. Results: Item-level models achieved the highest performance for both dropout (F1-score = 0.87, Brier score = 0.0529, balanced accuracy = 0.88) and treatment non-response (F1-score = 0.60, Brier score = 0.1646, balanced accuracy = 0.72) prediction. Items reflecting cognitive and bodily dimensions, respectively, emerged as key predictors. Conclusion: This study demonstrates the clinical value of using item-level data to enhance predictive modeling for dropout and treatment non-response and the potential to provide actionable insights for clinical practice. Integrating such models into clinical feedback systems could help identify at-risk patients and reduce dropout and non–response rates.Keywords: dropout, treatment non-response, prevention, machine learning, prediction.