Individual Contributions As Intrinsic Exploration Scaffolds For Multi-agent Reinforcement Learning

Abstract

In multi-agent reinforcement learning (MARL), effective exploration is critical, especially in sparse reward environments. Although introducing global intrinsic rewards can foster exploration in such settings, it often complicates credit assignment among agents. To address this difficulty, we propose Individual Contributions as intrinsic Exploration Scaffolds (ICES), a novel approach to motivate exploration by assessing each agent's contribution from a global view. In particular, ICES constructs exploration scaffolds with Bayesian surprise, leveraging global transition information during centralized training. These scaffolds, used only in training, help to guide individual agents towards actions that significantly impact the global latent state transitions. Additionally, ICES separates exploration policies from exploitation policies, enabling the former to utilize privileged global information during training. Extensive experiments on cooperative benchmark tasks with sparse rewards, in

Stats

• citations0
• S2 citations—
• github stars24
• HF likes0
• heat score2.80
• arxiv keyli2024individual

Related papers

• Coordinated Exploration Via Intrinsic Rewards For Multi-agent Reinforcement Learning (2019)0.00
• Influence-based Reinforcement Learning For Intrinsically-motivated Agents (2021)0.00
• Imagine, Initialize, And Explore: An Effective Exploration Method In Multi-agent Reinforcement Learning (2024)2.26
• Exploration With Unreliable Intrinsic Reward In Multi-agent Reinforcement Learning (2019)0.00
• Curiosity-driven Multi-agent Exploration With Mixed Objectives (2022)0.00
• MIR: Efficient Exploration In Episodic Multi-agent Reinforcement Learning Via Mutual Intrinsic Reward (2025)0.00
• LIGS: Learnable Intrinsic-reward Generation Selection For Multi-agent Learning (2021)0.00
• The Impact Of Intrinsic Rewards On Exploration In Reinforcement Learning (2025)0.00