Is A Good Representation Sufficient For Sample Efficient Reinforcement Learning?

Abstract

Modern deep learning methods provide effective means to learn good representations. However, is a good representation itself sufficient for sample efficient reinforcement learning? This question has largely been studied only with respect to (worst-case) approximation error, in the more classical approximate dynamic programming literature. With regards to the statistical viewpoint, this question is largely unexplored, and the extant body of literature mainly focuses on conditions which permit sample efficient reinforcement learning with little understanding of what are necessary conditions for efficient reinforcement learning. This work shows that, from the statistical viewpoint, the situation is far subtler than suggested by the more traditional approximation viewpoint, where the requirements on the representation that suffice for sample efficient RL are even more stringent. Our main results provide sharp thresholds for reinforcement learning methods, showing that there are hard limi

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keydu2019is

Related papers

• On The Generalization Of Representations In Reinforcement Learning (2022)0.00
• Sample-efficient Reinforcement Learning Is Feasible For Linearly Realizable Mdps With Limited Revisiting (2021)0.00
• Which Mutual-information Representation Learning Objectives Are Sufficient For Control? (2021)0.00
• How Transferable Are The Representations Learned By Deep Q Agents? (2020)0.00
• A Survey Of State Representation Learning For Deep Reinforcement Learning (2025)0.00
• Representations For Stable Off-policy Reinforcement Learning (2020)0.00
• Learning Temporally-consistent Representations For Data-efficient Reinforcement Learning (2021)0.00
• Unsupervised Representation Learning In Deep Reinforcement Learning: A Review (2022)9.59