On Improving Model-free Algorithms For Decentralized Multi-agent Reinforcement Learning

Abstract

Multi-agent reinforcement learning (MARL) algorithms often suffer from an exponential sample complexity dependence on the number of agents, a phenomenon known as *the curse of multiagents*. In this paper, we address this challenge by investigating sample-efficient model-free algorithms in *decentralized* MARL, and aim to improve existing algorithms along this line. For learning (coarse) correlated equilibria in general-sum Markov games, we propose *stage-based* V-learning algorithms that significantly simplify the algorithmic design and analysis of recent works, and circumvent a rather complicated no-*weighted*-regret bandit subroutine. For learning Nash equilibria in Markov potential games, we propose an independent policy gradient algorithm with a decentralized momentum-based variance reduction technique. All our algorithms are decentralized in that each agent can make decisions based on only its local information. Neither communication nor centralized coordination is required during

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keymao2021on

Related papers

• Breaking The Curse Of Multiagency: Provably Efficient Decentralized Multi-agent RL With Function Approximation (2023)0.00
• V-learning -- A Simple, Efficient, Decentralized Algorithm For Multiagent RL (2021)0.00
• Fully Decentralized Multi-agent Reinforcement Learning With Networked Agents (2018)0.00
• Incentivize Without Bonus: Provably Efficient Model-based Online Multi-agent RL For Markov Games (2025)0.00
• Finite-sample Analysis For Decentralized Batch Multi-agent Reinforcement Learning With Networked Agents (2018)10.07
• On The Complexity Of Multi-agent Decision Making: From Learning In Games To Partial Monitoring (2023)0.00
• Mean-field Multi-agent Reinforcement Learning: A Decentralized Network Approach (2021)0.00
• Breaking The Curse Of Multiagency In Robust Multi-agent Reinforcement Learning (2024)0.00