Sample-efficient Robust Multi-agent Reinforcement Learning In The Face Of Environmental Uncertainty

Abstract

To overcome the sim-to-real gap in reinforcement learning (RL), learned policies must maintain robustness against environmental uncertainties. While robust RL has been widely studied in single-agent regimes, in multi-agent environments, the problem remains understudied -- despite the fact that the problems posed by environmental uncertainties are often exacerbated by strategic interactions. This work focuses on learning in distributionally robust Markov games (RMGs), a robust variant of standard Markov games, wherein each agent aims to learn a policy that maximizes its own worst-case performance when the deployed environment deviates within its own prescribed uncertainty set. This results in a set of robust equilibrium strategies for all agents that align with classic notions of game-theoretic equilibria. Assuming a non-adaptive sampling mechanism from a generative model, we propose a sample-efficient model-based algorithm (DRNVI) with finite-sample complexity guarantees for learning r

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyshi2024sample

Related papers

• Sample-efficient Distributionally Robust Multi-agent Reinforcement Learning Via Online Interaction (2025)0.00
• Robust Cooperative Multi-agent Reinforcement Learning:a Mean-field Type Game Perspective (2024)0.00
• Minimax-optimal Multi-agent Robust Reinforcement Learning (2024)0.00
• Breaking The Curse Of Multiagency In Robust Multi-agent Reinforcement Learning (2024)0.00
• The Curious Price Of Distributional Robustness In Reinforcement Learning With A Generative Model (2023)0.00
• Sample Complexity Of Robust Reinforcement Learning With A Generative Model (2021)0.00
• On Practical Robust Reinforcement Learning: Practical Uncertainty Set And Double-agent Algorithm (2023)3.58
• Feasible Adversarial Robust Reinforcement Learning For Underspecified Environments (2022)0.00