Efficient Episodic Memory Utilization Of Cooperative Multi-agent Reinforcement Learning
2024 · Hyungho Na, Yunkyeong Seo, Il-Chul Moon
Abstract
In cooperative multi-agent reinforcement learning (MARL), agents aim to achieve a common goal, such as defeating enemies or scoring a goal. Existing MARL algorithms are effective but still require significant learning time and often get trapped in local optima by complex tasks, subsequently failing to discover a goal-reaching policy. To address this, we introduce Efficient episodic Memory Utilization (EMU) for MARL, with two primary objectives: (a) accelerating reinforcement learning by leveraging semantically coherent memory from an episodic buffer and (b) selectively promoting desirable transitions to prevent local convergence. To achieve (a), EMU incorporates a trainable encoder/decoder structure alongside MARL, creating coherent memory embeddings that facilitate exploratory memory recall. To achieve (b), EMU introduces a novel reward structure called episodic incentive based on the desirability of states. This reward improves the TD target in Q-learning and acts as an additional in
Authors
(none)
Tags
Stats
Related papers
- Episodic Multi-agent Reinforcement Learning With Curiosity-driven Exploration (2021)0.00
- Prioritized Guidance For Efficient Multi-agent Reinforcement Learning Exploration (2019)0.00
- MIR: Efficient Exploration In Episodic Multi-agent Reinforcement Learning Via Mutual Intrinsic Reward (2025)0.00
- Higher Replay Ratio Empowers Sample-efficient Multi-agent Reinforcement Learning (2024)0.00
- Boosting Value Decomposition Via Unit-wise Attentive State Representation For Cooperative Multi-agent Reinforcement Learning (2023)0.00
- Exploiting Semantic Epsilon Greedy Exploration Strategy In Multi-agent Reinforcement Learning (2022)0.00
- Benchmarking Multi-agent Deep Reinforcement Learning Algorithms In Cooperative Tasks (2020)0.00
- Ensemble Value Functions For Efficient Exploration In Multi-agent Reinforcement Learning (2023)0.00