MACCA: Offline Multi-agent Reinforcement Learning With Causal Credit Assignment

Abstract

Offline Multi-agent Reinforcement Learning (MARL) is valuable in scenarios where online interaction is impractical or risky. While independent learning in MARL offers flexibility and scalability, accurately assigning credit to individual agents in offline settings poses challenges because interactions with an environment are prohibited. In this paper, we propose a new framework, namely Multi-Agent Causal Credit Assignment (MACCA), to address credit assignment in the offline MARL setting. Our approach, MACCA, characterizing the generative process as a Dynamic Bayesian Network, captures relationships between environmental variables, states, actions, and rewards. Estimating this model on offline data, MACCA can learn each agent's contribution by analyzing the causal relationship of their individual rewards, ensuring accurate and interpretable credit assignment. Additionally, the modularity of our approach allows it to seamlessly integrate with various offline MARL methods. Theoretically,

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keywang2023macca

Related papers

• Asynchronous Credit Assignment For Multi-agent Reinforcement Learning (2024)0.00
• Discovering Causality For Efficient Cooperation In Multi-agent Environments (2023)0.00
• Shapley Counterfactual Credits For Multi-agent Reinforcement Learning (2021)12.40
• Challenges In Credit Assignment For Multi-agent Reinforcement Learning In Open Agent Systems (2025)0.00
• Cooperative Game-theoretic Credit Assignment For Multi-agent Policy Gradients Via The Core (2025)0.00
• Learning To Communicate Using Counterfactual Reasoning (2020)0.00
• QLLM: Do We Really Need A Mixing Network For Credit Assignment In Multi-agent Reinforcement Learning? (2025)0.00
• MACIE: Multi-agent Causal Intelligence Explainer For Collective Behavior Understanding (2025)2.26