Inferring Latent Temporal Sparse Coordination Graph For Multi-agent Reinforcement Learning

Abstract

Effective agent coordination is crucial in cooperative Multi-Agent Reinforcement Learning (MARL). While agent cooperation can be represented by graph structures, prevailing graph learning methods in MARL are limited. They rely solely on one-step observations, neglecting crucial historical experiences, leading to deficient graphs that foster redundant or detrimental information exchanges. Additionally, high computational demands for action-pair calculations in dense graphs impede scalability. To address these challenges, we propose inferring a Latent Temporal Sparse Coordination Graph (LTS-CG) for MARL. The LTS-CG leverages agents' historical observations to calculate an agent-pair probability matrix, where a sparse graph is sampled from and used for knowledge exchange between agents, thereby simultaneously capturing agent dependencies and relation uncertainty. The computational complexity of this procedure is only related to the number of agents. This graph learning process is further

Stats

• citations7
• S2 citations—
• github stars0
• HF likes0
• heat score6.77
• arxiv keyduan2024inferring

Related papers

• Group-aware Coordination Graph For Multi-agent Reinforcement Learning (2024)0.00
• GCS: Graph-based Coordination Strategy For Multi-agent Reinforcement Learning (2022)0.00
• Coordination-driven Learning In Multi-agent Problem Spaces (2018)0.00
• Asynchronous Cooperative Multi-agent Reinforcement Learning With Limited Communication (2025)0.00
• Context-aware Sparse Deep Coordination Graphs (2021)0.00
• Hierarchical Deep Multiagent Reinforcement Learning With Temporal Abstraction (2018)0.00
• Learning To Advise And Learning From Advice In Cooperative Multi-agent Reinforcement Learning (2022)0.00
• Distributed Multi-agent Reinforcement Learning Based On Graph-induced Local Value Functions (2022)4.52