Low-dimensional State And Action Representation Learning With MDP Homomorphism Metrics

Abstract

Deep Reinforcement Learning has shown its ability in solving complicated problems directly from high-dimensional observations. However, in end-to-end settings, Reinforcement Learning algorithms are not sample-efficient and requires long training times and quantities of data. In this work, we proposed a framework for sample-efficient Reinforcement Learning that take advantage of state and action representations to transform a high-dimensional problem into a low-dimensional one. Moreover, we seek to find the optimal policy mapping latent states to latent actions. Because now the policy is learned on abstract representations, we enforce, using auxiliary loss functions, the lifting of such policy to the original problem domain. Results show that the novel framework can efficiently learn low-dimensional and interpretable state and action representations and the optimal latent policy.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keybotteghi2021low

Related papers

• Using Forwards-backwards Models To Approximate MDP Homomorphisms (2022)0.00
• Deepmdp: Learning Continuous Latent Space Models For Representation Learning (2019)0.00
• Learning Good State And Action Representations Via Tensor Decomposition (2021)2.26
• Model-free Representation Learning And Exploration In Low-rank Mdps (2021)0.00
• Achieving Sample And Computational Efficient Reinforcement Learning By Action Space Reduction Via Grouping (2023)0.00
• Representation Learning For Efficient Deep Multi-agent Reinforcement Learning (2024)0.00
• Learning The Minimum Action Distance (2025)0.00
• Kinematic State Abstraction And Provably Efficient Rich-observation Reinforcement Learning (2019)0.00