Reinforcement Learning In Dynamic Treatment Regimes Needs Critical Reexamination

Abstract

In the rapidly changing healthcare landscape, the implementation of offline reinforcement learning (RL) in dynamic treatment regimes (DTRs) presents a mix of unprecedented opportunities and challenges. This position paper offers a critical examination of the current status of offline RL in the context of DTRs. We argue for a reassessment of applying RL in DTRs, citing concerns such as inconsistent and potentially inconclusive evaluation metrics, the absence of naive and supervised learning baselines, and the diverse choice of RL formulation in existing research. Through a case study with more than 17,000 evaluation experiments using a publicly available Sepsis dataset, we demonstrate that the performance of RL algorithms can significantly vary with changes in evaluation metrics and Markov Decision Process (MDP) formulations. Surprisingly, it is observed that in some instances, RL algorithms can be surpassed by random baselines subjected to policy evaluation methods and reward design. T

Stats

• citations0
• S2 citations—
• github stars14
• HF likes0
• heat score2.35
• arxiv keyluo2024reinforcement

Related papers

• POLAR: A Pessimistic Model-based Policy Learning Algorithm For Dynamic Treatment Regimes (2025)0.00
• Federated Offline Reinforcement Learning (2022)0.00
• Reinforcement Learning Enhanced Online Adaptive Clinical Decision Support Via Digital Twin Powered Policy And Treatment Effect Optimized Reward (2025)0.00
• Stable CDE Autoencoders With Acuity Regularization For Offline Reinforcement Learning In Sepsis Treatment (2025)0.00
• Did We Personalize? Assessing Personalization By An Online Reinforcement Learning Algorithm Using Resampling (2023)4.52
• Deep Reinforcement Learning For Clinical Decision Support: A Brief Survey (2019)0.00
• Offline Reinforcement Learning With Differential Privacy (2022)0.00
• An Empirical Study Of Representation Learning For Reinforcement Learning In Healthcare (2020)0.00