Robust Cooperative Multi-agent Reinforcement Learning:a Mean-field Type Game Perspective

Abstract

In this paper, we study the problem of robust cooperative multi-agent reinforcement learning (RL) where a large number of cooperative agents with distributed information aim to learn policies in the presence of *stochastic* and *non-stochastic* uncertainties whose distributions are respectively known and unknown. Focusing on policy optimization that accounts for both types of uncertainties, we formulate the problem in a worst-case (minimax) framework, which is is intractable in general. Thus, we focus on the Linear Quadratic setting to derive benchmark solutions. First, since no standard theory exists for this problem due to the distributed information structure, we utilize the Mean-Field Type Game (MFTG) paradigm to establish guarantees on the solution quality in the sense of achieved Nash equilibrium of the MFTG. This in turn allows us to compare the performance against the corresponding original robust multi-agent control problem. Then, we propose a Receding-horizon Gradient Descent

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyzaman2024robust

Related papers

• Sample-efficient Robust Multi-agent Reinforcement Learning In The Face Of Environmental Uncertainty (2024)0.00
• Minimax-optimal Multi-agent Robust Reinforcement Learning (2024)0.00
• Breaking The Curse Of Multiagency In Robust Multi-agent Reinforcement Learning (2024)0.00
• Efficient Model-based Multi-agent Mean-field Reinforcement Learning (2021)0.00
• Sample-efficient Distributionally Robust Multi-agent Reinforcement Learning Via Online Interaction (2025)0.00
• Minimax-optimal Multi-agent RL In Markov Games With A Generative Model (2022)2.26
• Robust Multi-agent Reinforcement Learning With State Uncertainty (2023)0.00
• Decentralized Mean Field Games (2021)3.58