Byzantine Robust Cooperative Multi-agent Reinforcement Learning As A Bayesian Game

Abstract

In this study, we explore the robustness of cooperative multi-agent reinforcement learning (c-MARL) against Byzantine failures, where any agent can enact arbitrary, worst-case actions due to malfunction or adversarial attack. To address the uncertainty that any agent can be adversarial, we propose a Bayesian Adversarial Robust Dec-POMDP (BARDec-POMDP) framework, which views Byzantine adversaries as nature-dictated types, represented by a separate transition. This allows agents to learn policies grounded on their posterior beliefs about the type of other agents, fostering collaboration with identified allies and minimizing vulnerability to adversarial manipulation. We define the optimal solution to the BARDec-POMDP as an ex post robust Bayesian Markov perfect equilibrium, which we proof to exist and weakly dominates the equilibrium of previous robust MARL approaches. To realize this equilibrium, we put forward a two-timescale actor-critic algorithm with almost sure convergence under spe

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyli2023byzantine

Related papers

• Breaking The Curse Of Multiagency In Robust Multi-agent Reinforcement Learning (2024)0.00
• Robust Multi-agent Reinforcement Learning With State Uncertainty (2023)0.00
• Fault Tolerant Multi-agent Learning With Adversarial Budget Constraints (2025)0.00
• Attacking C-marl More Effectively: A Data Driven Approach (2022)0.00
• Attacking Cooperative Multi-agent Reinforcement Learning By Adversarial Minority Influence (2023)0.00
• Empirical Study On Robustness And Resilience In Cooperative Multi-agent Reinforcement Learning (2025)0.00
• Robust Multi-agent Reinforcement Learning Via Adversarial Regularization: Theoretical Foundation And Stable Algorithms (2023)2.98
• Adversarial Attacks In Consensus-based Multi-agent Reinforcement Learning (2021)0.00