Regularize! Don't Mix: Multi-agent Reinforcement Learning Without Explicit Centralized Structures
2021 · Chapman Siu, Jason Traish, Richard Yi da Xu
Abstract
We propose using regularization for Multi-Agent Reinforcement Learning rather than learning explicit cooperative structures called \{\em Multi-Agent Regularized Q-learning\} (MARQ). Many MARL approaches leverage centralized structures in order to exploit global state information or removing communication constraints when the agents act in a decentralized manner. Instead of learning redundant structures which is removed during agent execution, we propose instead to leverage shared experiences of the agents to regularize the individual policies in order to promote structured exploration. We examine several different approaches to how MARQ can either explicitly or implicitly regularize our policies in a multi-agent setting. MARQ aims to address these limitations in the MARL context through applying regularization constraints which can correct bias in off-policy out-of-distribution agent experiences and promote diverse exploration. Our algorithm is evaluated on several benchmark multi-agen
Authors
(none)
Tags
Stats
Related papers
- Robust Multi-agent Reinforcement Learning By Mutual Information Regularization (2023)0.00
- Revisiting Some Common Practices In Cooperative Multi-agent Reinforcement Learning (2022)0.00
- Promoting Coordination Through Policy Regularization In Multi-agent Deep Reinforcement Learning (2019)0.00
- MA2QL: A Minimalist Approach To Fully Decentralized Multi-agent Reinforcement Learning (2022)0.00
- Mitigating Relative Over-generalization In Multi-agent Reinforcement Learning (2024)0.00
- Robust Multi-agent Reinforcement Learning Via Adversarial Regularization: Theoretical Foundation And Stable Algorithms (2023)2.98
- Comadice: Offline Cooperative Multi-agent Reinforcement Learning With Stationary Distribution Shift Regularization (2024)0.00
- Hypermarl: Adaptive Hypernetworks For Multi-agent RL (2024)0.00