On The Challenges Of Using Reinforcement Learning In Precision Drug Dosing: Delay And Prolongedness Of Action Effects

Abstract

Drug dosing is an important application of AI, which can be formulated as a Reinforcement Learning (RL) problem. In this paper, we identify two major challenges of using RL for drug dosing: delayed and prolonged effects of administering medications, which break the Markov assumption of the RL framework. We focus on prolongedness and define PAE-POMDP (Prolonged Action Effect-Partially Observable Markov Decision Process), a subclass of POMDPs in which the Markov assumption does not hold specifically due to prolonged effects of actions. Motivated by the pharmacology literature, we propose a simple and effective approach to converting drug dosing PAE-POMDPs into MDPs, enabling the use of the existing RL algorithms to solve such problems. We validate the proposed approach on a toy task, and a challenging glucose control task, for which we devise a clinically-inspired reward function. Our results demonstrate that: (1) the proposed method to restore the Markov assumption leads to significant

Stats

• citations3
• S2 citations—
• github stars0
• HF likes0
• heat score4.52
• arxiv keybasu2023on

Related papers

• An Improved Strategy For Blood Glucose Control Using Multi-step Deep Reinforcement Learning (2024)5.84
• Reinforcement Learning In Dynamic Treatment Regimes Needs Critical Reexamination (2024)2.35
• Revisiting State Augmentation Methods For Reinforcement Learning With Stochastic Delays (2021)10.35
• Omg-rl:offline Model-based Guided Reward Learning For Heparin Treatment (2024)0.00
• Application Of Deep Reinforcement Learning To Event-triggered Control For Networked Artificial Pancreas Systems (2026)0.00
• PDRL: Multi-agent Based Reinforcement Learning For Predictive Monitoring (2023)0.00
• Optimal Decision-making In Mixed-agent Partially Observable Stochastic Environments Via Reinforcement Learning (2019)0.00
• Optimizing The Long-term Average Reward For Continuing Mdps: A Technical Report (2021)0.00