Towards Efficient Collaboration Via Graph Modeling In Reinforcement Learning

Abstract

In multi-agent reinforcement learning, a commonly considered paradigm is centralized training with decentralized execution. However, in this framework, decentralized execution restricts the development of coordinated policies due to the local observation limitation. In this paper, we consider the cooperation among neighboring agents during execution and formulate their interactions as a graph. Thus, we introduce a novel encoder-decoder architecture named Factor-based Multi-Agent Transformer (\(f\)-MAT) that utilizes a transformer to enable communication between neighboring agents during both training and execution. By dividing agents into different overlapping groups and representing each group with a factor, \(f\)-MAT achieves efficient message passing and parallel action generation through factor-based attention layers. Empirical results in networked systems such as traffic scheduling and power control demonstrate that \(f\)-MAT achieves superior performance compared to strong baseli

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyfan2024towards

Related papers

• GCS: Graph-based Coordination Strategy For Multi-agent Reinforcement Learning (2022)0.00
• Counterfactual Multi-agent Reinforcement Learning With Graph Convolution Communication (2020)0.00
• Distributed Multi-agent Reinforcement Learning Based On Graph-induced Local Value Functions (2022)4.52
• Bridging Training And Execution Via Dynamic Directed Graph-based Communication In Cooperative Multi-agent Systems (2024)0.00
• Group-aware Coordination Graph For Multi-agent Reinforcement Learning (2024)0.00
• Efficient Policy Generation In Multi-agent Systems Via Hypergraph Neural Network (2022)0.00
• More Centralized Training, Still Decentralized Execution: Multi-agent Conditional Policy Factorization (2022)0.00
• Inferring Latent Temporal Sparse Coordination Graph For Multi-agent Reinforcement Learning (2024)6.77