What Is The Solution For State-adversarial Multi-agent Reinforcement Learning?

Abstract

Various methods for Multi-Agent Reinforcement Learning (MARL) have been developed with the assumption that agents' policies are based on accurate state information. However, policies learned through Deep Reinforcement Learning (DRL) are susceptible to adversarial state perturbation attacks. In this work, we propose a State-Adversarial Markov Game (SAMG) and make the first attempt to investigate different solution concepts of MARL under state uncertainties. Our analysis shows that the commonly used solution concepts of optimal agent policy and robust Nash equilibrium do not always exist in SAMGs. To circumvent this difficulty, we consider a new solution concept called robust agent policy, where agents aim to maximize the worst-case expected state value. We prove the existence of robust agent policy for finite state and finite action SAMGs. Additionally, we propose a Robust Multi-Agent Adversarial Actor-Critic (RMA3C) algorithm to learn robust policies for MARL agents under state uncerta

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyhan2022what

Related papers

• Robust Multi-agent Reinforcement Learning With State Uncertainty (2023)0.00
• Breaking The Curse Of Multiagency In Robust Multi-agent Reinforcement Learning (2024)0.00
• Camouflage Adversarial Attacks On Multiple Agent Systems (2024)0.00
• Robust Multi-agent Reinforcement Learning Via Adversarial Regularization: Theoretical Foundation And Stable Algorithms (2023)2.98
• Enhancing The Robustness Of QMIX Against State-adversarial Attacks (2023)9.76
• SUB-PLAY: Adversarial Policies Against Partially Observed Multi-agent Reinforcement Learning Systems (2024)0.00
• Byzantine Robust Cooperative Multi-agent Reinforcement Learning As A Bayesian Game (2023)0.00
• Sok: Adversarial Machine Learning Attacks And Defences In Multi-agent Reinforcement Learning (2023)10.74