GCS: Graph-based Coordination Strategy For Multi-agent Reinforcement Learning

Abstract

Many real-world scenarios involve a team of agents that have to coordinate their policies to achieve a shared goal. Previous studies mainly focus on decentralized control to maximize a common reward and barely consider the coordination among control policies, which is critical in dynamic and complicated environments. In this work, we propose factorizing the joint team policy into a graph generator and graph-based coordinated policy to enable coordinated behaviours among agents. The graph generator adopts an encoder-decoder framework that outputs directed acyclic graphs (DAGs) to capture the underlying dynamic decision structure. We also apply the DAGness-constrained and DAG depth-constrained optimization in the graph generator to balance efficiency and performance. The graph-based coordinated policy exploits the generated decision structure. The graph generator and coordinated policy are trained simultaneously to maximize the discounted return. Empirical evaluations on Collaborative Ga

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyruan2022gcs

Related papers

• Inferring Latent Temporal Sparse Coordination Graph For Multi-agent Reinforcement Learning (2024)6.77
• Group-aware Coordination Graph For Multi-agent Reinforcement Learning (2024)0.00
• Promoting Coordination Through Policy Regularization In Multi-agent Deep Reinforcement Learning (2019)0.00
• Towards Efficient Collaboration Via Graph Modeling In Reinforcement Learning (2024)0.00
• Counterfactual Multi-agent Reinforcement Learning With Graph Convolution Communication (2020)0.00
• Signal Instructed Coordination In Cooperative Multi-agent Reinforcement Learning (2019)4.52
• Learning Multi-agent Coordination Through Connectivity-driven Communication (2020)8.60
• Learning Generalizable Risk-sensitive Policies To Coordinate In Decentralized Multi-agent General-sum Games (2022)0.00