Influence-based Reinforcement Learning For Intrinsically-motivated Agents

Abstract

Discovering successful coordinated behaviors is a central challenge in Multi-Agent Reinforcement Learning (MARL) since it requires exploring a joint action space that grows exponentially with the number of agents. In this paper, we propose a mechanism for achieving sufficient exploration and coordination in a team of agents. Specifically, agents are rewarded for contributing to a more diversified team behavior by employing proper intrinsic motivation functions. To learn meaningful coordination protocols, we structure agents' interactions by introducing a novel framework, where at each timestep, an agent simulates counterfactual rollouts of its policy and, through a sequence of computations, assesses the gap between other agents' current behaviors and their targets. Actions that minimize the gap are considered highly influential and are rewarded. We evaluate our approach on a set of challenging tasks with sparse rewards and partial observability that require learning complex cooperative

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyfayad2021influence

Related papers

• Social Influence As Intrinsic Motivation For Multi-agent Deep Reinforcement Learning (2018)0.00
• Coordinated Exploration Via Intrinsic Rewards For Multi-agent Reinforcement Learning (2019)0.00
• Situation-dependent Causal Influence-based Cooperative Multi-agent Reinforcement Learning (2023)5.24
• Joint Intrinsic Motivation For Coordinated Exploration In Multi-agent Deep Reinforcement Learning (2024)0.00
• Individual Contributions As Intrinsic Exploration Scaffolds For Multi-agent Reinforcement Learning (2024)2.80
• A Principle Of Targeted Intervention For Multi-agent Reinforcement Learning (2025)0.00
• Robust Multi-agent Reinforcement Learning With Social Empowerment For Coordination And Communication (2020)0.00
• Discovering Individual Rewards In Collective Behavior Through Inverse Multi-agent Reinforcement Learning (2023)0.00