Learning Sparse Representations In Reinforcement Learning With Sparse Coding

Abstract

A variety of representation learning approaches have been investigated for reinforcement learning; much less attention, however, has been given to investigating the utility of sparse coding. Outside of reinforcement learning, sparse coding representations have been widely used, with non-convex objectives that result in discriminative representations. In this work, we develop a supervised sparse coding objective for policy evaluation. Despite the non-convexity of this objective, we prove that all local minima are global minima, making the approach amenable to simple optimization strategies. We empirically show that it is key to use a supervised objective, rather than the more straightforward unsupervised sparse coding approach. We compare the learned representations to a canonical fixed sparse representation, called tile-coding, demonstrating that the sparse coding representation outperforms a wide variety of tilecoding representations.

Stats

• citations14
• S2 citations—
• github stars0
• HF likes0
• heat score8.82
• arxiv keyle2017learning

Related papers

• The Utility Of Sparse Representations For Control In Reinforcement Learning (2018)10.48
• Learning Sparse Representations Incrementally In Deep Reinforcement Learning (2019)0.00
• Optimal Control With Natural Images: Efficient Reinforcement Learning Using Overcomplete Sparse Codes (2024)0.00
• Locally Constrained Representations In Reinforcement Learning (2022)0.00
• Shaping Sparse Rewards In Reinforcement Learning: A Semi-supervised Approach (2025)0.00
• Representation-driven Reinforcement Learning (2023)0.00
• Online Reinforcement Learning With Sparse Rewards Through An Active Inference Capsule (2021)0.00
• Multi-objective Model-based Policy Search For Data-efficient Learning With Sparse Rewards (2018)0.00