Provably Efficient Cooperative Multi-agent Reinforcement Learning With Function Approximation

Abstract

Reinforcement learning in cooperative multi-agent settings has recently advanced significantly in its scope, with applications in cooperative estimation for advertising, dynamic treatment regimes, distributed control, and federated learning. In this paper, we discuss the problem of cooperative multi-agent RL with function approximation, where a group of agents communicates with each other to jointly solve an episodic MDP. We demonstrate that via careful message-passing and cooperative value iteration, it is possible to achieve near-optimal no-regret learning even with a fixed constant communication budget. Next, we demonstrate that even in heterogeneous cooperative settings, it is possible to achieve Pareto-optimal no-regret learning with limited communication. Our work generalizes several ideas from the multi-agent contextual and multi-armed bandit literature to MDPs and reinforcement learning.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keydubey2021provably

Related papers

• Cooperative Multi-agent Reinforcement Learning: Asynchronous Communication And Linear Function Approximation (2023)0.00
• Breaking The Curse Of Multiagency: Provably Efficient Decentralized Multi-agent RL With Function Approximation (2023)0.00
• Resilient Consensus-based Multi-agent Reinforcement Learning With Function Approximation (2021)0.00
• Strategically Robust Multi-agent Reinforcement Learning With Linear Function Approximation (2026)0.00
• Provably Efficient Reinforcement Learning With Linear Function Approximation (2019)11.76
• Distributed Value Function Approximation For Collaborative Multi-agent Reinforcement Learning (2020)8.60
• Robust Cooperative Multi-agent Reinforcement Learning:a Mean-field Type Game Perspective (2024)0.00
• Optimization For Reinforcement Learning: From Single Agent To Cooperative Agents (2019)14.62