Context-aware Sparse Deep Coordination Graphs

Abstract

Learning sparse coordination graphs adaptive to the coordination dynamics among agents is a long-standing problem in cooperative multi-agent learning. This paper studies this problem and proposes a novel method using the variance of payoff functions to construct context-aware sparse coordination topologies. We theoretically consolidate our method by proving that the smaller the variance of payoff functions is, the less likely action selection will change after removing the corresponding edge. Moreover, we propose to learn action representations to effectively reduce the influence of payoff functions' estimation errors on graph construction. To empirically evaluate our method, we present the Multi-Agent COordination (MACO) benchmark by collecting classic coordination problems in the literature, increasing their difficulty, and classifying them into different types. We carry out a case study and experiments on the MACO and StarCraft II micromanagement benchmark to demonstrate the dynamic

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keywang2021context

Related papers

• Inferring Latent Temporal Sparse Coordination Graph For Multi-agent Reinforcement Learning (2024)6.77
• GCS: Graph-based Coordination Strategy For Multi-agent Reinforcement Learning (2022)0.00
• Coordination-driven Learning In Multi-agent Problem Spaces (2018)0.00
• A Structured Prediction Approach For Generalization In Cooperative Multi-agent Reinforcement Learning (2019)0.00
• Coach-player Multi-agent Reinforcement Learning For Dynamic Team Composition (2021)0.00
• Learning Generalizable Risk-sensitive Policies To Coordinate In Decentralized Multi-agent General-sum Games (2022)0.00
• Coot: Learning To Coordinate In-context With Coordination Transformers (2025)0.00
• Context-aware Bayesian Network Actor-critic Methods For Cooperative Multi-agent Reinforcement Learning (2023)0.00