Value Enhancement Of Reinforcement Learning Via Efficient And Robust Trust Region Optimization

Abstract

Reinforcement learning (RL) is a powerful machine learning technique that enables an intelligent agent to learn an optimal policy that maximizes the cumulative rewards in sequential decision making. Most of methods in the existing literature are developed in \textit\{online\} settings where the data are easy to collect or simulate. Motivated by high stake domains such as mobile health studies with limited and pre-collected data, in this paper, we study \textit\{offline\} reinforcement learning methods. To efficiently use these datasets for policy optimization, we propose a novel value enhancement method to improve the performance of a given initial policy computed by existing state-of-the-art RL algorithms. Specifically, when the initial policy is not consistent, our method will output a policy whose value is no worse and often better than that of the initial policy. When the initial policy is consistent, under some mild conditions, our method will yield a policy whose value converges

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyshi2023value

Related papers

• Uncertainty-aware Policy Optimization: A Robust, Adaptive Trust Region Approach (2020)0.00
• Trust Region Policy Optimisation In Multi-agent Reinforcement Learning (2021)0.00
• Embedding Safety Into RL: A New Take On Trust Region Methods (2024)0.00
• Average-reward Reinforcement Learning With Trust Region Methods (2021)0.00
• Statistically Efficient Advantage Learning For Offline Reinforcement Learning In Infinite Horizons (2022)0.00
• Adaptive Trust Region Policy Optimization: Global Convergence And Faster Rates For Regularized Mdps (2019)12.10
• Confidence-conditioned Value Functions For Offline Reinforcement Learning (2022)0.00
• Reinforcement Learning With Non-ergodic Reward Increments: Robustness Via Ergodicity Transformations (2023)0.00