Sample Complexity Of Offline Reinforcement Learning With Deep Relu Networks

Abstract

Offline reinforcement learning (RL) leverages previously collected data for policy optimization without any further active exploration. Despite the recent interest in this problem, its theoretical results in neural network function approximation settings remain elusive. In this paper, we study the statistical theory of offline RL with deep ReLU network function approximation. In particular, we establish the sample complexity of \(n = \tilde\{\mathcal\{O\}\}( H^\{4 + 4 \frac\{d\}\{\alpha\}\} \kappa_\{\mu\}^\{1 + \frac\{d\}\{\alpha\}\} \epsilon^\{-2 - 2\frac\{d\}\{\alpha\}\} )\) for offline RL with deep ReLU networks, where \(\kappa_\{\mu\}\) is a measure of distributional shift, \{\(H = (1-\gamma)^\{-1\}\) is the effective horizon length\}, \(d\) is the dimension of the state-action space, \(\alpha\) is a (possibly fractional) smoothness parameter of the underlying Markov decision process (MDP), and \(\epsilon\) is a user-specified error. Notably, our sample complexity holds under two n

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keynguyentang2021sample

Related papers

• Distributionally Robust Model-based Offline Reinforcement Learning With Near-optimal Sample Complexity (2022)0.00
• Importance Of Empirical Sample Complexity Analysis For Offline Reinforcement Learning (2021)0.00
• Distributionally Robust Offline Reinforcement Learning With Linear Function Approximation (2022)0.00
• On The Sample Complexity Of Vanilla Model-based Offline Reinforcement Learning With Dependent Samples (2023)2.26
• Neural Network Approximation For Pessimistic Offline Reinforcement Learning (2023)0.00
• What Are The Statistical Limits Of Offline RL With Linear Function Approximation? (2020)0.00
• Sample Complexity Of Offline Distributionally Robust Linear Markov Decision Processes (2024)0.00
• Infinite-horizon Offline Reinforcement Learning With Linear Function Approximation: Curse Of Dimensionality And Algorithm (2021)0.00