MUSBO: Model-based Uncertainty Regularized And Sample Efficient Batch Optimization For Deployment Constrained Reinforcement Learning
2021 · Dijia Su, Jason D. Lee, John M. Mulvey, et al.
Abstract
In many contemporary applications such as healthcare, finance, robotics, and recommendation systems, continuous deployment of new policies for data collection and online learning is either cost ineffective or impractical. We consider a setting that lies between pure offline reinforcement learning (RL) and pure online RL called deployment constrained RL in which the number of policy deployments for data sampling is limited. To solve this challenging task, we propose a new algorithmic learning framework called Model-based Uncertainty regularized and Sample Efficient Batch Optimization (MUSBO). Our framework discovers novel and high quality samples for each deployment to enable efficient data collection. During each offline training session, we bootstrap the policy update by quantifying the amount of uncertainty within our collected data. In the high support region (low uncertainty), we encourage our policy by taking an aggressive update. In the low support region (high uncertainty) when
Authors
(none)
Tags
Stats
Related papers
- Deployment-efficient Reinforcement Learning Via Model-based Offline Optimization (2020)0.00
- Enhancing Offline Model-based RL Via Active Model Selection: A Bayesian Optimization Perspective (2025)0.00
- Deep Model-based Reinforcement Learning Via Estimated Uncertainty And Conservative Policy Optimization (2019)0.00
- Robust Model-free Reinforcement Learning With Multi-objective Bayesian Optimization (2019)11.08
- Distributionally Robust Model-based Offline Reinforcement Learning With Near-optimal Sample Complexity (2022)0.00
- Towards Robust Offline-to-online Reinforcement Learning Via Uncertainty And Smoothness (2023)5.24
- Online Robust Reinforcement Learning With Model Uncertainty (2021)0.00
- Overcoming Model Bias For Robust Offline Deep Reinforcement Learning (2020)11.58