Coordinated Exploration Via Intrinsic Rewards For Multi-agent Reinforcement Learning

Abstract

Solving tasks with sparse rewards is one of the most important challenges in reinforcement learning. In the single-agent setting, this challenge is addressed by introducing intrinsic rewards that motivate agents to explore unseen regions of their state spaces; however, applying these techniques naively to the multi-agent setting results in agents exploring independently, without any coordination among themselves. Exploration in cooperative multi-agent settings can be accelerated and improved if agents coordinate their exploration. In this paper we introduce a framework for designing intrinsic rewards which consider what other agents have explored such that the agents can coordinate. Then, we develop an approach for learning how to dynamically select between several exploration modalities to maximize extrinsic rewards. Concretely, we formulate the approach as a hierarchical policy where a high-level controller selects among sets of policies trained on diverse intrinsic rewards and the l

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyiqbal2019coordinated

Related papers

• Curiosity-driven Multi-agent Exploration With Mixed Objectives (2022)0.00
• Exploration With Unreliable Intrinsic Reward In Multi-agent Reinforcement Learning (2019)0.00
• Influence-based Reinforcement Learning For Intrinsically-motivated Agents (2021)0.00
• Individual Contributions As Intrinsic Exploration Scaffolds For Multi-agent Reinforcement Learning (2024)2.80
• Joint Intrinsic Motivation For Coordinated Exploration In Multi-agent Deep Reinforcement Learning (2024)0.00
• Never Explore Repeatedly In Multi-agent Reinforcement Learning (2023)0.00
• Curiosity-driven Exploration In Sparse-reward Multi-agent Reinforcement Learning (2023)0.00
• Intrinsic Reward Policy Optimization For Sparse-reward Environments (2026)0.00