Stable CDE Autoencoders With Acuity Regularization For Offline Reinforcement Learning In Sepsis Treatment
2025 · Yue Gao
Abstract
Effective reinforcement learning (RL) for sepsis treatment depends on learning stable, clinically meaningful state representations from irregular ICU time series. While previous works have explored representation learning for this task, the critical challenge of training instability in sequential representations and its detrimental impact on policy performance has been overlooked. This work demonstrates that Controlled Differential Equations (CDE) state representation can achieve strong RL policies when two key factors are met: (1) ensuring training stability through early stopping or stabilization methods, and (2) enforcing acuity-aware representations by correlation regularization with clinical scores (SOFA, SAPS-II, OASIS). Experiments on the MIMIC-III sepsis cohort reveal that stable CDE autoencoder produces representations strongly correlated with acuity scores and enables RL policies with superior performance (WIS return \(> 0.9\)). In contrast, unstable CDE representation leads
Authors
(none)
Tags
Stats
Related papers
- An Empirical Study Of Representation Learning For Reinforcement Learning In Healthcare (2020)0.00
- Reinforcement Learning In Dynamic Treatment Regimes Needs Critical Reexamination (2024)2.35
- Reinforcement Learning Enhanced Online Adaptive Clinical Decision Support Via Digital Twin Powered Policy And Treatment Effect Optimized Reward (2025)0.00
- Semi-supervised Off Policy Reinforcement Learning (2020)0.00
- Learning From Sparse Offline Datasets Via Conservative Density Estimation (2024)0.00
- Statistically Efficient Advantage Learning For Offline Reinforcement Learning In Infinite Horizons (2022)0.00
- Adaptive Multi-agent Deep Reinforcement Learning For Timely Healthcare Interventions (2023)0.00
- Reinforcement Learning With Adaptive Regularization For Safe Control Of Critical Systems (2024)0.00