Learning Implicit Credit Assignment For Cooperative Multi-agent Reinforcement Learning

Abstract

We present a multi-agent actor-critic method that aims to implicitly address the credit assignment problem under fully cooperative settings. Our key motivation is that credit assignment among agents may not require an explicit formulation as long as (1) the policy gradients derived from a centralized critic carry sufficient information for the decentralized agents to maximize their joint action value through optimal cooperation and (2) a sustained level of exploration is enforced throughout training. Under the centralized training with decentralized execution (CTDE) paradigm, we achieve the former by formulating the centralized critic as a hypernetwork such that a latent state representation is integrated into the policy gradients through its multiplicative association with the stochastic policies; to achieve the latter, we derive a simple technique called adaptive entropy regularization where magnitudes of the entropy gradients are dynamically rescaled based on the current policy stoc

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyzhou2020learning

Related papers

• Actor-attention-critic For Multi-agent Reinforcement Learning (2018)0.00
• Shapley Counterfactual Credits For Multi-agent Reinforcement Learning (2021)12.40
• Multi-agent Actor-critic For Mixed Cooperative-competitive Environments (2017)0.00
• Cooperative Game-theoretic Credit Assignment For Multi-agent Policy Gradients Via The Core (2025)0.00
• Local Advantage Actor-critic For Robust Multi-agent Deep Reinforcement Learning (2021)7.81
• RACA: Relation-aware Credit Assignment For Ad-hoc Cooperation In Multi-agent Deep Reinforcement Learning (2022)3.58
• Context-aware Bayesian Network Actor-critic Methods For Cooperative Multi-agent Reinforcement Learning (2023)0.00
• Counterfactual Multi-agent Policy Gradients (2017)0.00