Is Centralized Training With Decentralized Execution Framework Centralized Enough For MARL?
2023 · Yihe Zhou, Shunyu Liu, Yunpeng Qing, et al.
Abstract
Centralized Training with Decentralized Execution (CTDE) has recently emerged as a popular framework for cooperative Multi-Agent Reinforcement Learning (MARL), where agents can use additional global state information to guide training in a centralized way and make their own decisions only based on decentralized local policies. Despite the encouraging results achieved, CTDE makes an independence assumption on agent policies, which limits agents to adopt global cooperative information from each other during centralized training. Therefore, we argue that existing CTDE methods cannot fully utilize global information for training, leading to an inefficient joint-policy exploration and even suboptimal results. In this paper, we introduce a novel Centralized Advising and Decentralized Pruning (CADP) framework for multi-agent reinforcement learning, that not only enables an efficacious message exchange among agents during training but also guarantees the independent policies for execution. Fir
Authors
(none)
Tags
Stats
Related papers
- CTDS: Centralized Teacher With Decentralized Student For Multi-agent Reinforcement Learning (2022)0.00
- Tacit Learning With Adaptive Information Selection For Cooperative Multi-agent Reinforcement Learning (2024)0.00
- GTDE: Grouped Training With Decentralized Execution For Multi-agent Actor-critic (2024)3.58
- An Initial Introduction To Cooperative Multi-agent Reinforcement Learning (2024)0.00
- Towards Global Optimality In Cooperative MARL With The Transformation And Distillation Framework (2022)0.00
- Taming Multi-agent Reinforcement Learning With Estimator Variance Reduction (2022)0.00
- Multi-agent Guided Policy Optimization (2025)0.00
- From Explicit Communication To Tacit Cooperation:a Novel Paradigm For Cooperative MARL (2023)3.58