Shapley Q-value: A Local Reward Approach To Solve Global Reward Games
2019 · Jianhong Wang, Yuan Zhang, Tae-Kyun Kim, et al.
Abstract
Cooperative game is a critical research area in the multi-agent reinforcement learning (MARL). Global reward game is a subclass of cooperative games, where all agents aim to maximize the global reward. Credit assignment is an important problem studied in the global reward game. Most of previous works stood by the view of non-cooperative-game theoretical framework with the shared reward approach, i.e., each agent being assigned a shared global reward directly. This, however, may give each agent an inaccurate reward on its contribution to the group, which could cause inefficient learning. To deal with this problem, we i) introduce a cooperative-game theoretical framework called extended convex game (ECG) that is a superset of global reward game, and ii) propose a local reward approach called Shapley Q-value. Shapley Q-value is able to distribute the global reward, reflecting each agent's own contribution in contrast to the shared reward approach. Moreover, we derive an MARL algorithm cal
Authors
(none)
Tags
Stats
Related papers
- SHAQ: Incorporating Shapley Value Theory Into Multi-agent Q-learning (2021)0.00
- Cooperative Game-theoretic Credit Assignment For Multi-agent Policy Gradients Via The Core (2025)0.00
- Collective Explainable AI: Explaining Cooperative Strategies And Agent Contribution In Multiagent Reinforcement Learning With Shapley Values (2021)0.00
- Locality Matters: A Scalable Value Decomposition Approach For Cooperative Multi-agent Reinforcement Learning (2021)0.00
- Shapley Counterfactual Credits For Multi-agent Reinforcement Learning (2021)12.40
- Efficiently Quantifying Individual Agent Importance In Cooperative MARL (2023)0.00
- Q-value Path Decomposition For Deep Multiagent Reinforcement Learning (2020)0.00
- Adaptive Value Decomposition With Greedy Marginal Contribution Computation For Cooperative Multi-agent Reinforcement Learning (2023)3.58