Exploiting Semantic Epsilon Greedy Exploration Strategy In Multi-agent Reinforcement Learning

Abstract

Multi-agent reinforcement learning (MARL) can model many real world applications. However, many MARL approaches rely on epsilon greedy for exploration, which may discourage visiting advantageous states in hard scenarios. In this paper, we propose a new approach QMIX(SEG) for tackling MARL. It makes use of the value function factorization method QMIX to train per-agent policies and a novel Semantic Epsilon Greedy (SEG) exploration strategy. SEG is a simple extension to the conventional epsilon greedy exploration strategy, yet it is experimentally shown to greatly improve the performance of MARL. We first cluster actions into groups of actions with similar effects and then use the groups in a bi-level epsilon greedy exploration hierarchy for action selection. We argue that SEG facilitates semantic exploration by exploring in the space of groups of actions, which have richer semantic meanings than atomic actions. Experiments show that QMIX(SEG) largely outperforms QMIX and leads to strong

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keytse2022exploiting

Related papers

• MESA: Cooperative Meta-exploration In Multi-agent Learning Through Exploiting State-action Space Structure (2024)2.26
• Graph Exploration For Effective Multi-agent Q-learning (2023)5.24
• Episodic Multi-agent Reinforcement Learning With Curiosity-driven Exploration (2021)0.00
• Ensemble Value Functions For Efficient Exploration In Multi-agent Reinforcement Learning (2023)0.00
• REMAX: Relational Representation For Multi-agent Exploration (2020)2.26
• Prioritized Guidance For Efficient Multi-agent Reinforcement Learning Exploration (2019)0.00
• Individual Specialization In Multi-task Environments With Multiagent Reinforcement Learners (2019)0.00
• Incentivize Without Bonus: Provably Efficient Model-based Online Multi-agent RL For Markov Games (2025)0.00