PMIC: Improving Multi-agent Reinforcement Learning With Progressive Mutual Information Collaboration
2022 · Pengyi Li, Hongyao Tang, Tianpei Yang, et al.
Abstract
Learning to collaborate is critical in Multi-Agent Reinforcement Learning (MARL). Previous works promote collaboration by maximizing the correlation of agents' behaviors, which is typically characterized by Mutual Information (MI) in different forms. However, we reveal sub-optimal collaborative behaviors also emerge with strong correlations, and simply maximizing the MI can, surprisingly, hinder the learning towards better collaboration. To address this issue, we propose a novel MARL framework, called Progressive Mutual Information Collaboration (PMIC), for more effective MI-driven collaboration. PMIC uses a new collaboration criterion measured by the MI between global states and joint actions. Based on this criterion, the key idea of PMIC is maximizing the MI associated with superior collaborative behaviors and minimizing the MI associated with inferior ones. The two MI objectives play complementary roles by facilitating better collaborations while avoiding falling into sub-optimal on
Authors
(none)
Tags
Stats
Related papers
- Situation-dependent Causal Influence-based Cooperative Multi-agent Reinforcement Learning (2023)5.24
- Iterated Reasoning With Mutual Information In Cooperative And Byzantine Decentralized Teaming (2022)0.00
- Robust Multi-agent Reinforcement Learning By Mutual Information Regularization (2023)0.00
- Cautiously-optimistic Knowledge Sharing For Cooperative Multi-agent Reinforcement Learning (2023)5.84
- A Variational Approach To Mutual Information-based Coordination For Multi-agent Reinforcement Learning (2023)0.00
- Robust Multi-agent Reinforcement Learning With Social Empowerment For Coordination And Communication (2020)0.00
- Efficient Distributed Framework For Collaborative Multi-agent Reinforcement Learning (2022)0.00
- Influence-based Reinforcement Learning For Intrinsically-motivated Agents (2021)0.00