Shapley Counterfactual Credits For Multi-agent Reinforcement Learning
2021 · Jiahui Li, Kun Kuang, Baoxiang Wang, et al.
Abstract
Centralized Training with Decentralized Execution (CTDE) has been a popular paradigm in cooperative Multi-Agent Reinforcement Learning (MARL) settings and is widely used in many real applications. One of the major challenges in the training process is credit assignment, which aims to deduce the contributions of each agent according to the global rewards. Existing credit assignment methods focus on either decomposing the joint value function into individual value functions or measuring the impact of local observations and actions on the global value function. These approaches lack a thorough consideration of the complicated interactions among multiple agents, leading to an unsuitable assignment of credit and subsequently mediocre results on MARL. We propose Shapley Counterfactual Credit Assignment, a novel method for explicit credit assignment which accounts for the coalition of agents. Specifically, Shapley Value and its desired properties are leveraged in deep MARL to credit any combi
Authors
(none)
Tags
Stats
Related papers
- STAS: Spatial-temporal Return Decomposition For Multi-agent Reinforcement Learning (2023)0.00
- Learning Implicit Credit Assignment For Cooperative Multi-agent Reinforcement Learning (2020)0.00
- Cooperative Game-theoretic Credit Assignment For Multi-agent Policy Gradients Via The Core (2025)0.00
- Asynchronous Credit Assignment For Multi-agent Reinforcement Learning (2024)0.00
- QLLM: Do We Really Need A Mixing Network For Credit Assignment In Multi-agent Reinforcement Learning? (2025)0.00
- MACCA: Offline Multi-agent Reinforcement Learning With Causal Credit Assignment (2023)0.00
- Adaptive Value Decomposition With Greedy Marginal Contribution Computation For Cooperative Multi-agent Reinforcement Learning (2023)3.58
- Assigning Credit With Partial Reward Decoupling In Multi-agent Proximal Policy Optimization (2024)0.00