Federated Ensemble-directed Offline Reinforcement Learning

Abstract

We consider the problem of federated offline reinforcement learning (RL), a scenario under which distributed learning agents must collaboratively learn a high-quality control policy only using small pre-collected datasets generated according to different unknown behavior policies. Na\"\{i\}vely combining a standard offline RL approach with a standard federated learning approach to solve this problem can lead to poorly performing policies. In response, we develop the Federated Ensemble-Directed Offline Reinforcement Learning Algorithm (FEDORA), which distills the collective wisdom of the clients using an ensemble learning approach. We develop the FEDORA codebase to utilize distributed compute resources on a federated learning platform. We show that FEDORA significantly outperforms other approaches, including offline RL over the combined data pool, in various complex continuous control environments and real-world datasets. Finally, we demonstrate the performance of FEDORA in the real-wor

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyrengarajan2023federated

Related papers

• Federated Offline Reinforcement Learning: Collaborative Single-policy Coverage Suffices (2024)0.00
• Federated Offline Policy Optimization With Dual Regularization (2024)3.58
• Federated Ensemble Model-based Reinforcement Learning In Edge Computing (2021)11.58
• Momentum For The Win: Collaborative Federated Reinforcement Learning Across Heterogeneous Environments (2024)0.00
• Federated Offline Reinforcement Learning (2022)0.00
• Federated Offline Policy Learning (2023)0.00
• Fedhpd: Heterogeneous Federated Reinforcement Learning Via Policy Distillation (2025)2.26
• Provably Robust Federated Reinforcement Learning (2025)0.00