On The Generalization Of Representations In Reinforcement Learning

Abstract

In reinforcement learning, state representations are used to tractably deal with large problem spaces. State representations serve both to approximate the value function with few parameters, but also to generalize to newly encountered states. Their features may be learned implicitly (as part of a neural network) or explicitly (for example, the successor representation of \citet\{dayan1993improving\}). While the approximation properties of representations are reasonably well-understood, a precise characterization of how and when these representations generalize is lacking. In this work, we address this gap and provide an informative bound on the generalization error arising from a specific state representation. This bound is based on the notion of effective dimension which measures the degree to which knowing the value at one state informs the value at other states. Our bound applies to any state representation and quantifies the natural tension between representations that generalize w

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keylan2022on

Related papers

• Measuring And Characterizing Generalization In Deep Reinforcement Learning (2018)9.76
• Is A Good Representation Sufficient For Sample Efficient Reinforcement Learning? (2019)0.00
• A Survey Of State Representation Learning For Deep Reinforcement Learning (2025)0.00
• Locally Constrained Representations In Reinforcement Learning (2022)0.00
• Bootstrapped Representations In Reinforcement Learning (2023)0.00
• A Survey Analyzing Generalization In Deep Reinforcement Learning (2024)0.00
• Representations For Stable Off-policy Reinforcement Learning (2020)0.00
• Proper Laplacian Representation Learning (2023)0.00