Offline Reinforcement Learning With Differential Privacy

Abstract

The offline reinforcement learning (RL) problem is often motivated by the need to learn data-driven decision policies in financial, legal and healthcare applications. However, the learned policy could retain sensitive information of individuals in the training data (e.g., treatment and outcome of patients), thus susceptible to various privacy risks. We design offline RL algorithms with differential privacy guarantees which provably prevent such risks. These algorithms also enjoy strong instance-dependent learning bounds under both tabular and linear Markov decision process (MDP) settings. Our theory and simulation suggest that the privacy guarantee comes at (almost) no drop in utility comparing to the non-private counterpart for a medium-size dataset.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyqiao2022offline

Related papers

• Preserving Expert-level Privacy In Offline Reinforcement Learning (2024)0.00
• Near-optimal Differentially Private Reinforcement Learning (2022)0.00
• Privorl: Differentially Private Synthetic Dataset For Offline Reinforcement Learning (2025)0.00
• Kl-regularization Itself Is Differentially Private In Bandits And RLHF (2025)0.00
• Local Differential Privacy For Regret Minimization In Reinforcement Learning (2020)0.00
• Privacy-preserving Reinforcement Learning From Human Feedback Via Decoupled Reward Modeling (2026)0.00
• Differentially Private Policy Evaluation (2016)0.00
• Privacy Preserving Reinforcement Learning For Population Processes (2024)0.00