Experience Augmentation: Boosting And Accelerating Off-policy Multi-agent Reinforcement Learning
2020 · Zhenhui Ye, Yining Chen, Guanghua Song, et al.
Abstract
Exploration of the high-dimensional state action space is one of the biggest challenges in Reinforcement Learning (RL), especially in multi-agent domain. We present a novel technique called Experience Augmentation, which enables a time-efficient and boosted learning based on a fast, fair and thorough exploration to the environment. It can be combined with arbitrary off-policy MARL algorithms and is applicable to either homogeneous or heterogeneous environments. We demonstrate our approach by combining it with MADDPG and verifing the performance in two homogeneous and one heterogeneous environments. In the best performing scenario, the MADDPG with experience augmentation reaches to the convergence reward of vanilla MADDPG with 1/4 realistic time, and its convergence beats the original model by a significant margin. Our ablation studies show that experience augmentation is a crucial ingredient which accelerates the training process and boosts the convergence.
Authors
(none)
Tags
Stats
Related papers
- Policy Augmentation: An Exploration Strategy For Faster Convergence Of Deep Reinforcement Learning Algorithms (2021)2.26
- Prioritized Guidance For Efficient Multi-agent Reinforcement Learning Exploration (2019)0.00
- Autoeg: Automated Experience Grafting For Off-policy Deep Reinforcement Learning (2020)0.00
- A Further Exploration Of Deep Multi-agent Reinforcement Learning With Hybrid Action Space (2022)5.84
- MESA: Cooperative Meta-exploration In Multi-agent Learning Through Exploiting State-action Space Structure (2024)2.26
- Exploiting Semantic Epsilon Greedy Exploration Strategy In Multi-agent Reinforcement Learning (2022)0.00
- Stabilising Experience Replay For Deep Multi-agent Reinforcement Learning (2017)0.00
- Enhancing Sample Efficiency In Multi-agent RL With Uncertainty Quantification And Selective Exploration (2025)0.00