Centralized Model And Exploration Policy For Multi-agent RL

Abstract

Reinforcement learning (RL) in partially observable, fully cooperative multi-agent settings (Dec-POMDPs) can in principle be used to address many real-world challenges such as controlling a swarm of rescue robots or a team of quadcopters. However, Dec-POMDPs are significantly harder to solve than single-agent problems, with the former being NEXP-complete and the latter, MDPs, being just P-complete. Hence, current RL algorithms for Dec-POMDPs suffer from poor sample complexity, which greatly reduces their applicability to practical problems where environment interaction is costly. Our key insight is that using just a polynomial number of samples, one can learn a centralized model that generalizes across different policies. We can then optimize the policy within the learned model instead of the true system, without requiring additional environment interactions. We also learn a centralized exploration policy within our model that learns to collect additional data in state-action regions w

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyzhang2021centralized

Related papers

• A New Framework For Multi-agent Reinforcement Learning -- Centralized Training And Exploration With Decentralized Execution Via Policy Distillation (2019)0.00
• Deep Decentralized Multi-task Multi-agent Reinforcement Learning Under Partial Observability (2017)0.00
• Optimal Decision-making In Mixed-agent Partially Observable Stochastic Environments Via Reinforcement Learning (2019)0.00
• Centralized Cooperative Exploration Policy For Continuous Control Tasks (2023)0.00
• Multi-agent Off-policy Actor-critic Reinforcement Learning For Partially Observable Environments (2024)2.26
• Macro-action-based Multi-agent/robot Deep Reinforcement Learning Under Partial Observability (2022)5.84
• Scalable Centralized Deep Multi-agent Reinforcement Learning Via Policy Gradients (2018)0.00
• Reducing Overestimation Bias In Multi-agent Domains Using Double Centralized Critics (2019)0.00