Spectral Decomposition Representation For Reinforcement Learning

Abstract

Representation learning often plays a critical role in reinforcement learning by managing the curse of dimensionality. A representative class of algorithms exploits a spectral decomposition of the stochastic transition dynamics to construct representations that enjoy strong theoretical properties in an idealized setting. However, current spectral methods suffer from limited applicability because they are constructed for state-only aggregation and derived from a policy-dependent transition kernel, without considering the issue of exploration. To address these issues, we propose an alternative spectral method, Spectral Decomposition Representation (SPEDER), that extracts a state-action abstraction from the dynamics without inducing spurious dependence on the data collection policy, while also balancing the exploration-versus-exploitation trade-off during learning. A theoretical analysis establishes the sample efficiency of the proposed algorithm in both the online and offline settings. I

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyren2022spectral

Related papers

• Spectral Representation-based Reinforcement Learning (2025)0.00
• Spectral Bellman Method: Unifying Representation And Exploration In RL (2025)0.00
• Representation-driven Reinforcement Learning (2023)0.00
• Understanding Self-predictive Learning For Reinforcement Learning (2022)0.00
• A Survey Of State Representation Learning For Deep Reinforcement Learning (2025)0.00
• Data-efficient Reinforcement Learning With Self-predictive Representations (2020)0.00
• Experimental Results : Reinforcement Learning Of Pomdps Using Spectral Methods (2017)0.00
• Unsupervised Representation Learning In Deep Reinforcement Learning: A Review (2022)9.59