Data Efficient Training For Reinforcement Learning With Adaptive Behavior Policy Sharing
2020 · Ge Liu, Rui Wu, Heng-Tze Cheng, et al.
Abstract
Deep Reinforcement Learning (RL) is proven powerful for decision making in simulated environments. However, training deep RL model is challenging in real world applications such as production-scale health-care or recommender systems because of the expensiveness of interaction and limitation of budget at deployment. One aspect of the data inefficiency comes from the expensive hyper-parameter tuning when optimizing deep neural networks. We propose Adaptive Behavior Policy Sharing (ABPS), a data-efficient training algorithm that allows sharing of experience collected by behavior policy that is adaptively selected from a pool of agents trained with an ensemble of hyper-parameters. We further extend ABPS to evolve hyper-parameters during training by hybridizing ABPS with an adapted version of Population Based Training (ABPS-PBT). We conduct experiments with multiple Atari games with up to 16 hyper-parameter/architecture setups. ABPS achieves superior overall performance, reduced variance on
Authors
(none)
Tags
Stats
Related papers
- Sample-efficient Automated Deep Reinforcement Learning (2020)0.00
- Erl-re\(^2\): Efficient Evolutionary Reinforcement Learning With Shared State Representation And Individual Policy Representation (2022)0.00
- Learning A Subspace Of Policies For Online Adaptation In Reinforcement Learning (2021)0.00
- Hyperparameter Tuning For Deep Reinforcement Learning Applications (2022)0.00
- Parameter Sharing Deep Deterministic Policy Gradient For Cooperative Multi-agent Reinforcement Learning (2017)0.00
- Policy Agnostic RL: Offline RL And Online RL Fine-tuning Of Any Class And Backbone (2024)0.00
- Automatic Tuning Of Hyper-parameters Of Reinforcement Learning Algorithms Using Bayesian Optimization With Behavioral Cloning (2021)0.00
- Generalized Population-based Training For Hyperparameter Optimization In Reinforcement Learning (2024)9.59