Reinforcement Learning for Chronic Care Pathway Optimization: A Unified Framework across Three Clinical Goal Types
Discuss this preprint
Start a discussion What are Sciety discussions?Listed in
This article is not in any list yet, why not save it to one of your lists.Abstract
Objective
Chronic care requires sequential treatment under competing biomarker, safety, and cost constraints, yet clinical goal structures differ across diseases. We asked whether one physiology-informed reinforcement learning (RL) paradigm adapts to heterogeneous chronic-care goals without disease-specific policy architectures.
Materials and Methods
We formalized a Type A/B/C clinical goal taxonomy (target cure, stable cruise, cycle completion) as a Physiology-Informed Markov Decision Process registry for gout, chronic kidney disease (CKD), and PCOS-mediated fertility treatment—each with PK/PD transitions, discrete actions, safety zones, and guideline doctor baselines. Unified BC → PPO training (GAE λ =0.95) on 500 simulated trajectories per disease. Evaluation: paired seeds ( N =50 primary; N =500 bootstrap 95% CIs), 10-seed robustness, ablation, literature sUA calibration, and out-of-distribution stress. McNemar/Wilcoxon with Benjamini–Hochberg FDR.
Results
PCOS (Type C, primary): PPO 72.0% vs. doctor 54.0% at N =50 (+18 percentage points; FDR-significant); at N =500, PPO 69.8% [65.6, 73.8] vs. doctor 52.8% [48.8, 57.2]. Gout (Type A): PPO non-inferior—88.0% vs. 90.0% (McNemar p =1.0). CKD (Type B): doctor 32.0%, BC/PPO 38.0%. Offline CQL 92.0% on gout trajectories. PK recalibration RMSE 97.4 µ mol/L ( r =0.809).
Conclusions
Shared BC → PPO training generalizes across three goal types without cross-disease weight sharing. PCOS supports RL for bounded cycles; gout confirms guideline non-inferiority; CKD illustrates cruise-control difficulty. This framework offers a reproducible foundation for chronic pathway optimization pending prospective validation.
Highlights
-
Type A/B/C taxonomy unifies chronic-care RL across three clinical goal structures
-
PCOS cycle completion +18pp vs doctor baseline (72% vs 54% at N =50)
-
Cross-disease PIMDP: gout non-inferior, CKD cruise-control stress test
-
BC → PPO generalizes without shared weights; open reproducible artifacts
