AIR: Unifying Individual And Collective Exploration In Cooperative Multi-agent Reinforcement Learning

Abstract

Exploration in cooperative multi-agent reinforcement learning (MARL) remains challenging for value-based agents due to the absence of an explicit policy. Existing approaches include individual exploration based on uncertainty towards the system and collective exploration through behavioral diversity among agents. However, the introduction of additional structures often leads to reduced training efficiency and infeasible integration of these methods. In this paper, we propose Adaptive exploration via Identity Recognition~(AIR), which consists of two adversarial components: a classifier that recognizes agent identities from their trajectories, and an action selector that adaptively adjusts the mode and degree of exploration. We theoretically prove that AIR can facilitate both individual and collective exploration during training, and experiments also demonstrate the efficiency and effectiveness of AIR across various tasks.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyzhou2024air

Related papers

• Individual Contributions As Intrinsic Exploration Scaffolds For Multi-agent Reinforcement Learning (2024)2.80
• Imagine, Initialize, And Explore: An Effective Exploration Method In Multi-agent Reinforcement Learning (2024)2.26
• Ensemble Value Functions For Efficient Exploration In Multi-agent Reinforcement Learning (2023)0.00
• Influence-based Reinforcement Learning For Intrinsically-motivated Agents (2021)0.00
• Modeling The Interaction Between Agents In Cooperative Multi-agent Reinforcement Learning (2021)0.00
• Individual Specialization In Multi-task Environments With Multiagent Reinforcement Learners (2019)0.00
• Policy Distillation And Value Matching In Multiagent Reinforcement Learning (2019)10.48
• MESA: Cooperative Meta-exploration In Multi-agent Learning Through Exploiting State-action Space Structure (2024)2.26