On The Linear Speedup Of Personalized Federated Reinforcement Learning With Shared Representations
2024 · Guojun Xiong, Shufan Wang, Daniel Jiang, et al.
Abstract
Federated reinforcement learning (FedRL) enables multiple agents to collaboratively learn a policy without sharing their local trajectories collected during agent-environment interactions. However, in practice, the environments faced by different agents are often heterogeneous, leading to poor performance by the single policy learned by existing FedRL algorithms on individual agents. In this paper, we take a further step and introduce a *personalized* FedRL framework (PFedRL) by taking advantage of possibly shared common structure among agents in heterogeneous environments. Specifically, we develop a class of PFedRL algorithms named PFedRL-Rep that learns (1) a shared feature representation collaboratively among all agents, and (2) an agent-specific weight vector personalized to its local environment. We analyze the convergence of PFedTD-Rep, a particular instance of the framework with temporal difference (TD) learning and linear representations. To the best of our knowledge, we are th
Authors
(none)
Tags
Stats
Related papers
- Fedhpd: Heterogeneous Federated Reinforcement Learning Via Policy Distillation (2025)2.26
- Momentum For The Win: Collaborative Federated Reinforcement Learning Across Heterogeneous Environments (2024)0.00
- Collaborative Value Function Estimation Under Model Mismatch: A Federated Temporal Difference Analysis (2025)0.00
- Federated Distributional Reinforcement Learning With Distributional Critic Regularization (2026)0.00
- Asynchronous Federated Reinforcement Learning With Policy Gradient Updates: Algorithm Design And Convergence Analysis (2024)0.00
- Finite-time Analysis Of On-policy Heterogeneous Federated Reinforcement Learning (2024)0.00
- Heterogeneity-aware Personalized Federated Learning Via Adaptive Dual-agent Reinforcement Learning (2025)0.00
- Federated Reinforcement Learning With Constraint Heterogeneity (2024)0.00