Fedhpd: Heterogeneous Federated Reinforcement Learning Via Policy Distillation

Abstract

Federated Reinforcement Learning (FedRL) improves sample efficiency while preserving privacy; however, most existing studies assume homogeneous agents, limiting its applicability in real-world scenarios. This paper investigates FedRL in black-box settings with heterogeneous agents, where each agent employs distinct policy networks and training configurations without disclosing their internal details. Knowledge Distillation (KD) is a promising method for facilitating knowledge sharing among heterogeneous models, but it faces challenges related to the scarcity of public datasets and limitations in knowledge representation when applied to FedRL. To address these challenges, we propose Federated Heterogeneous Policy Distillation (FedHPD), which solves the problem of heterogeneous FedRL by utilizing action probability distributions as a medium for knowledge sharing. We provide a theoretical analysis of FedHPD's convergence under standard assumptions. Extensive experiments corroborate that F

Stats

• citations1
• S2 citations—
• github stars0
• HF likes0
• heat score2.26
• arxiv keyjiang2025fedhpd

Related papers

• On The Linear Speedup Of Personalized Federated Reinforcement Learning With Shared Representations (2024)0.00
• Federated Distributional Reinforcement Learning With Distributional Critic Regularization (2026)0.00
• Federated Reinforcement Learning With Constraint Heterogeneity (2024)0.00
• Momentum For The Win: Collaborative Federated Reinforcement Learning Across Heterogeneous Environments (2024)0.00
• Finite-time Analysis Of On-policy Heterogeneous Federated Reinforcement Learning (2024)0.00
• Federated Reinforcement Distillation With Proxy Experience Memory (2019)3.58
• Asynchronous Federated Reinforcement Learning With Policy Gradient Updates: Algorithm Design And Convergence Analysis (2024)0.00
• Federated Offline Policy Optimization With Dual Regularization (2024)3.58