HRRT: A Holistic Renal Replacement Therapy Decision-Making Support System Using Hierarchical Reinforcement Learning

Renal replacement therapy (RRT) is a critical intervention for patients with acute kidney injury (AKI). However, clinical decision-making regarding the timing of initiation, modality selection, optimal ultrafiltration rate, and weaning criteria remains highly complex and exhibits significant practice variation. While RCTs have demonstrated that indiscriminate high-intensity RRT offers no benefit in unselected populations, they fail to guide adaptive strategies, leaving dynamic decision-making heavily reliant on empirical experience. Consequently, a substantial gap persists in delivering patientspecific recommendations, particularly for dynamically adjusting treatment in response to clinical progression. Therefore, we develop Hierarchical Reinforcement Learning for Renal Replacement Therapy (HRRT), a holistic clinical decision support system (CDSS) that covers the full decisionmaking process in RRT. 2,467 Intensive Care Unit (ICU) stays of 1,439 patients, within a cohort of patients with AKI from a US hospital, were used for training and internal testing of the model. The model’s performance was evaluated on two external validation sets, where we selected 1085 ICU stays of 1085 patients from Netherlands and 1230 stays of 845 patients from China. The estimated mortality rate decreased by 6.1 percentage points from 47.7% (95%CI: 45.2 – 50.0) to 41.6% (95%CI: 35.6 - 47.2) compared to clinician-led outcomes.