SVDE: Scalable Value-decomposition Exploration For Cooperative Multi-agent Reinforcement Learning
2023 · Shuhan Qi, Shuhao Zhang, Qiang Wang, et al.
Abstract
Value-decomposition methods, which reduce the difficulty of a multi-agent system by decomposing the joint state-action space into local observation-action spaces, have become popular in cooperative multi-agent reinforcement learning (MARL). However, value-decomposition methods still have the problems of tremendous sample consumption for training and lack of active exploration. In this paper, we propose a scalable value-decomposition exploration (SVDE) method, which includes a scalable training mechanism, intrinsic reward design, and explorative experience replay. The scalable training mechanism asynchronously decouples strategy learning with environmental interaction, so as to accelerate sample generation in a MapReduce manner. For the problem of lack of exploration, an intrinsic reward design and explorative experience replay are proposed, so as to enhance exploration to produce diverse samples and filter non-novel samples, respectively. Empirically, our method achieves the best perfo
Authors
(none)
Tags
Stats
Related papers
- Uneven: Universal Value Exploration For Multi-agent Reinforcement Learning (2020)0.00
- VDFD: Multi-agent Value Decomposition Framework With Disentangled World Model (2023)0.00
- Adaptive Value Decomposition With Greedy Marginal Contribution Computation For Cooperative Multi-agent Reinforcement Learning (2023)3.58
- Locality Matters: A Scalable Value Decomposition Approach For Cooperative Multi-agent Reinforcement Learning (2021)0.00
- MAVEN: Multi-agent Variational Exploration (2019)0.00
- Understanding Value Decomposition Algorithms In Deep Cooperative Multi-agent Reinforcement Learning (2022)0.00
- Boosting Value Decomposition Via Unit-wise Attentive State Representation For Cooperative Multi-agent Reinforcement Learning (2023)0.00
- Modeling The Interaction Between Agents In Cooperative Multi-agent Reinforcement Learning (2021)0.00