Spectral Representation-based Reinforcement Learning

Abstract

In real-world applications with large state and action spaces, reinforcement learning (RL) typically employs function approximations to represent core components like the policies, value functions, and dynamics models. Although powerful approximations such as neural networks offer great expressiveness, they often present theoretical ambiguities, suffer from optimization instability and exploration difficulty, and incur substantial computational costs in practice. In this paper, we introduce the perspective of spectral representations as a solution to address these difficulties in RL. Stemming from the spectral decomposition of the transition operator, this framework yields an effective abstraction of the system dynamics for subsequent policy optimization while also providing a clear theoretical characterization. We reveal how to construct spectral representations for transition operators that possess latent variable structures or energy-based structures, which implies different learnin

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keygao2025spectral

Related papers

• Spectral Decomposition Representation For Reinforcement Learning (2022)0.00
• Spectral Bellman Method: Unifying Representation And Exploration In RL (2025)0.00
• Representation-driven Reinforcement Learning (2023)0.00
• Balancing Interpretability And Performance In Reinforcement Learning: An Adaptive Spectral Based Linear Approach (2025)0.00
• Learning Symbolic Representations For Reinforcement Learning Of Non-markovian Behavior (2023)0.00
• Simplifying Model-based RL: Learning Representations, Latent-space Models, And Policies With One Objective (2022)0.00
• Reinforcement Learning In Rich-observation Mdps Using Spectral Methods (2016)0.00
• Rethinking Model-based, Policy-based, And Value-based Reinforcement Learning Via The Lens Of Representation Complexity (2023)2.26