Reducing Overestimation Bias In Multi-agent Domains Using Double Centralized Critics

Abstract

Many real world tasks require multiple agents to work together. Multi-agent reinforcement learning (RL) methods have been proposed in recent years to solve these tasks, but current methods often fail to efficiently learn policies. We thus investigate the presence of a common weakness in single-agent RL, namely value function overestimation bias, in the multi-agent setting. Based on our findings, we propose an approach that reduces this bias by using double centralized critics. We evaluate it on six mixed cooperative-competitive tasks, showing a significant advantage over current methods. Finally, we investigate the application of multi-agent methods to high-dimensional robotic tasks and show that our approach can be used to learn decentralized policies in this domain.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyackermann2019reducing

Related papers

• Multi-agent Actor-critic For Mixed Cooperative-competitive Environments (2017)0.00
• Contrasting Centralized And Decentralized Critics In Multi-agent Reinforcement Learning (2021)0.00
• A Deeper Understanding Of State-based Critics In Multi-agent Reinforcement Learning (2022)6.34
• Policy Distillation And Value Matching In Multiagent Reinforcement Learning (2019)10.48
• Actor-attention-critic For Multi-agent Reinforcement Learning (2018)0.00
• Cooperative And Competitive Biases For Multi-agent Reinforcement Learning (2021)2.26
• Scalable Centralized Deep Multi-agent Reinforcement Learning Via Policy Gradients (2018)0.00
• On Centralized Critics In Multi-agent Reinforcement Learning (2024)9.03