Discovering Causality For Efficient Cooperation In Multi-agent Environments

Abstract

In cooperative Multi-Agent Reinforcement Learning (MARL) agents are required to learn behaviours as a team to achieve a common goal. However, while learning a task, some agents may end up learning sub-optimal policies, not contributing to the objective of the team. Such agents are called lazy agents due to their non-cooperative behaviours that may arise from failing to understand whether they caused the rewards. As a consequence, we observe that the emergence of cooperative behaviours is not necessarily a byproduct of being able to solve a task as a team. In this paper, we investigate the applications of causality in MARL and how it can be applied in MARL to penalise these lazy agents. We observe that causality estimations can be used to improve the credit assignment to the agents and show how it can be leveraged to improve independent learning in MARL. Furthermore, we investigate how Amortized Causal Discovery can be used to automate causality detection within MARL environments. The r

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keypina2023discovering

Related papers

• A Roadmap Towards Improving Multi-agent Reinforcement Learning With Causal Discovery And Inference (2025)0.00
• Situation-dependent Causal Influence-based Cooperative Multi-agent Reinforcement Learning (2023)5.24
• Social Influence As Intrinsic Motivation For Multi-agent Deep Reinforcement Learning (2018)0.00
• Influence-based Reinforcement Learning For Intrinsically-motivated Agents (2021)0.00
• MACCA: Offline Multi-agent Reinforcement Learning With Causal Credit Assignment (2023)0.00
• MACIE: Multi-agent Causal Intelligence Explainer For Collective Behavior Understanding (2025)2.26
• Cautiously-optimistic Knowledge Sharing For Cooperative Multi-agent Reinforcement Learning (2023)5.84
• Inducing Cooperation Via Team Regret Minimization Based Multi-agent Deep Reinforcement Learning (2019)0.00