Episodic Multi-agent Reinforcement Learning With Curiosity-driven Exploration

Abstract

Efficient exploration in deep cooperative multi-agent reinforcement learning (MARL) still remains challenging in complex coordination problems. In this paper, we introduce a novel Episodic Multi-agent reinforcement learning with Curiosity-driven exploration, called EMC. We leverage an insight of popular factorized MARL algorithms that the "induced" individual Q-values, i.e., the individual utility functions used for local execution, are the embeddings of local action-observation histories, and can capture the interaction between agents due to reward backpropagation during centralized training. Therefore, we use prediction errors of individual Q-values as intrinsic rewards for coordinated exploration and utilize episodic memory to exploit explored informative experience to boost policy training. As the dynamics of an agent's individual Q-value function captures the novelty of states and the influence from other agents, our intrinsic reward can induce coordinated exploration to new or pr

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyzheng2021episodic

Related papers

• Wonder Wins Ways: Curiosity-driven Exploration Through Multi-agent Contextual Calibration (2025)0.00
• Efficient Episodic Memory Utilization Of Cooperative Multi-agent Reinforcement Learning (2024)0.00
• MIR: Efficient Exploration In Episodic Multi-agent Reinforcement Learning Via Mutual Intrinsic Reward (2025)0.00
• Graph Exploration For Effective Multi-agent Q-learning (2023)5.24
• Curiosity-driven Multi-agent Exploration With Mixed Objectives (2022)0.00
• Settling Decentralized Multi-agent Coordinated Exploration By Novelty Sharing (2024)4.52
• Ensemble Value Functions For Efficient Exploration In Multi-agent Reinforcement Learning (2023)0.00
• Exploiting Semantic Epsilon Greedy Exploration Strategy In Multi-agent Reinforcement Learning (2022)0.00