Batch Policy Learning In Average Reward Markov Decision Processes

Abstract

We consider the batch (off-line) policy learning problem in the infinite horizon Markov Decision Process. Motivated by mobile health applications, we focus on learning a policy that maximizes the long-term average reward. We propose a doubly robust estimator for the average reward and show that it achieves semiparametric efficiency. Further we develop an optimization algorithm to compute the optimal policy in a parameterized stochastic policy class. The performance of the estimated policy is measured by the difference between the optimal average reward in the policy class and the average reward of the estimated policy and we establish a finite-sample regret guarantee. The performance of the method is illustrated by simulation studies and an analysis of a mobile health study promoting physical activity.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyliao2020batch

Related papers

• Robust Batch Policy Learning In Markov Decision Processes (2020)0.00
• Stackelberg Batch Policy Learning (2023)0.00
• Learning And Planning In Average-reward Markov Decision Processes (2020)0.00
• Policy Gradient For Continuing Tasks In Non-stationary Markov Decision Processes (2020)0.00
• Optimal Sample Complexity For Average Reward Markov Decision Processes (2023)0.00
• Stochastic First-order Methods For Average-reward Markov Decision Processes (2022)3.58
• Doubly Robust Off-policy Value And Gradient Estimation For Deterministic Policies (2020)0.00
• Reinforcement Learning For Infinite-horizon Average-reward Linear Mdps Via Approximation By Discounted-reward Mdps (2024)0.00