Federated Offline Reinforcement Learning

Abstract

Evidence-based or data-driven dynamic treatment regimes are essential for personalized medicine, which can benefit from offline reinforcement learning (RL). Although massive healthcare data are available across medical institutions, they are prohibited from sharing due to privacy constraints. Besides, heterogeneity exists in different sites. As a result, federated offline RL algorithms are necessary and promising to deal with the problems. In this paper, we propose a multi-site Markov decision process model that allows for both homogeneous and heterogeneous effects across sites. The proposed model makes the analysis of the site-level features possible. We design the first federated policy optimization algorithm for offline RL with sample complexity. The proposed algorithm is communication-efficient, which requires only a single round of communication interaction by exchanging summary statistics. We give a theoretical guarantee for the proposed algorithm, where the suboptimality for the

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyzhou2022federated

Related papers

• Reinforcement Learning For Individual Optimal Policy From Heterogeneous Data (2025)0.00
• Federated Offline Reinforcement Learning: Collaborative Single-policy Coverage Suffices (2024)0.00
• Federated Offline Policy Optimization With Dual Regularization (2024)3.58
• Offline Reinforcement Learning With Differential Privacy (2022)0.00
• Federated Offline Policy Learning (2023)0.00
• Reinforcement Learning In Dynamic Treatment Regimes Needs Critical Reexamination (2024)2.35
• Federated Ensemble-directed Offline Reinforcement Learning (2023)0.00
• Deployment-efficient Reinforcement Learning Via Model-based Offline Optimization (2020)0.00