Provably Efficient Offline Goal-conditioned Reinforcement Learning With General Function Approximation And Single-policy Concentrability
2023 · Hanlin Zhu, Amy Zhang
Abstract
Goal-conditioned reinforcement learning (GCRL) refers to learning general-purpose skills that aim to reach diverse goals. In particular, offline GCRL only requires purely pre-collected datasets to perform training tasks without additional interactions with the environment. Although offline GCRL has become increasingly prevalent and many previous works have demonstrated its empirical success, the theoretical understanding of efficient offline GCRL algorithms is not well established, especially when the state space is huge and the offline dataset only covers the policy we aim to learn. In this paper, we provide a rigorous theoretical analysis of an existing empirically successful offline GCRL algorithm. We prove that under slight modification, this algorithm enjoys an \(\widetilde\{O\}(\text\{poly\}(1/\epsilon))\) sample complexity (where \(\epsilon\) is the desired suboptimality of the learned policy) with general function approximation thanks to the property of (semi-)strong convexity
Authors
(none)
Tags
Stats
Related papers
- Optimal Conservative Offline RL With General Function Approximation Via Augmented Lagrangian (2022)0.00
- Sample Efficient Active Algorithms For Offline Reinforcement Learning (2026)0.00
- SMORE: Score Models For Offline Goal-conditioned Reinforcement Learning (2023)0.00
- How Far I'll Go: Offline Goal-conditioned Reinforcement Learning Via \(f\)-advantage Regression (2022)0.00
- Distributionally Robust Offline Reinforcement Learning With Linear Function Approximation (2022)0.00
- What Are The Statistical Limits Of Offline RL With Linear Function Approximation? (2020)0.00
- Offline Reinforcement Learning: Fundamental Barriers For Value Function Approximation (2021)0.00
- Offline Reinforcement Learning With Realizability And Single-policy Concentrability (2022)0.00