Global Convergence Guarantees For Federated Policy Gradient Methods With Adversaries

Abstract

Federated Reinforcement Learning (FRL) allows multiple agents to collaboratively build a decision making policy without sharing raw trajectories. However, if a small fraction of these agents are adversarial, it can lead to catastrophic results. We propose a policy gradient based approach that is robust to adversarial agents which can send arbitrary values to the server. Under this setting, our results form the first global convergence guarantees with general parametrization. These results demonstrate resilience with adversaries, while achieving optimal sample complexity of order \(\tilde\{\mathcal\{O\}\}\left( \frac\{1\}\{N\epsilon^2\} \left( 1+ \frac\{f^2\}\{N\}\right)\right)\), where \(N\) is the total number of agents and \(f<N/2\) is the number of adversarial agents.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyganesh2024global

Related papers

• Asynchronous Federated Reinforcement Learning With Policy Gradient Updates: Algorithm Design And Convergence Analysis (2024)0.00
• Fault-tolerant Federated Reinforcement Learning With Theoretical Guarantee (2021)0.00
• Provably Robust Federated Reinforcement Learning (2025)0.00
• Momentum For The Win: Collaborative Federated Reinforcement Learning Across Heterogeneous Environments (2024)0.00
• Improved Communication Efficiency In Federated Natural Policy Gradient Via Admm-based Gradient Updates (2023)0.00
• The Gradient Convergence Bound Of Federated Multi-agent Reinforcement Learning With Efficient Communication (2021)0.00
• Federated Natural Policy Gradient And Actor Critic Methods For Multi-task Reinforcement Learning (2023)0.00
• On Global Convergence Rates For Federated Softmax Policy Gradient Under Heterogeneous Environments (2025)0.00