RACA: Relation-aware Credit Assignment For Ad-hoc Cooperation In Multi-agent Deep Reinforcement Learning

Abstract

In recent years, reinforcement learning has faced several challenges in the multi-agent domain, such as the credit assignment issue. Value function factorization emerges as a promising way to handle the credit assignment issue under the centralized training with decentralized execution (CTDE) paradigm. However, existing value function factorization methods cannot deal with ad-hoc cooperation, that is, adapting to new configurations of teammates at test time. Specifically, these methods do not explicitly utilize the relationship between agents and cannot adapt to different sizes of inputs. To address these limitations, we propose a novel method, called Relation-Aware Credit Assignment (RACA), which achieves zero-shot generalization in ad-hoc cooperation scenarios. RACA takes advantage of a graph-based relation encoder to encode the topological structure between agents. Furthermore, RACA utilizes an attention-based observation abstraction mechanism that can generalize to an arbitrary num

Stats

• citations2
• S2 citations—
• github stars0
• HF likes0
• heat score3.58
• arxiv keychen2022raca

Related papers

• Learning Implicit Credit Assignment For Cooperative Multi-agent Reinforcement Learning (2020)0.00
• Adaptive Value Decomposition With Greedy Marginal Contribution Computation For Cooperative Multi-agent Reinforcement Learning (2023)3.58
• Cooperative Game-theoretic Credit Assignment For Multi-agent Policy Gradients Via The Core (2025)0.00
• Cooperative Multi-agent Transfer Learning With Level-adaptive Credit Assignment (2021)0.00
• Asynchronous Credit Assignment For Multi-agent Reinforcement Learning (2024)0.00
• Assigning Credit With Partial Reward Decoupling In Multi-agent Proximal Policy Optimization (2024)0.00
• Shapley Counterfactual Credits For Multi-agent Reinforcement Learning (2021)12.40
• STAS: Spatial-temporal Return Decomposition For Multi-agent Reinforcement Learning (2023)0.00