Social Influence As Intrinsic Motivation For Multi-agent Deep Reinforcement Learning

Abstract

We propose a unified mechanism for achieving coordination and communication in Multi-Agent Reinforcement Learning (MARL), through rewarding agents for having causal influence over other agents' actions. Causal influence is assessed using counterfactual reasoning. At each timestep, an agent simulates alternate actions that it could have taken, and computes their effect on the behavior of other agents. Actions that lead to bigger changes in other agents' behavior are considered influential and are rewarded. We show that this is equivalent to rewarding agents for having high mutual information between their actions. Empirical results demonstrate that influence leads to enhanced coordination and communication in challenging social dilemma environments, dramatically increasing the learning curves of the deep RL agents, and leading to more meaningful learned communication protocols. The influence rewards for all agents can be computed in a decentralized way by enabling agents to learn a mode

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyjaques2018social

Related papers

• Influence-based Reinforcement Learning For Intrinsically-motivated Agents (2021)0.00
• Situation-dependent Causal Influence-based Cooperative Multi-agent Reinforcement Learning (2023)5.24
• Robust Multi-agent Reinforcement Learning With Social Empowerment For Coordination And Communication (2020)0.00
• Reward-sharing Relational Networks In Multi-agent Reinforcement Learning As A Framework For Emergent Behavior (2022)2.26
• Discovering Causality For Efficient Cooperation In Multi-agent Environments (2023)0.00
• Reliably Re-acting To Partner's Actions With The Social Intrinsic Motivation Of Transfer Empowerment (2022)0.00
• A Principle Of Targeted Intervention For Multi-agent Reinforcement Learning (2025)0.00
• Evolving Intrinsic Motivations For Altruistic Behavior (2018)2.26