On The Geometry Of Reinforcement Learning In Continuous State And Action Spaces

Abstract

Advances in reinforcement learning have led to its successful application in complex tasks with continuous state and action spaces. Despite these advances in practice, most theoretical work pertains to finite state and action spaces. We propose building a theoretical understanding of continuous state and action spaces by employing a geometric lens. Central to our work is the idea that the transition dynamics induce a low dimensional manifold of reachable states embedded in the high-dimensional nominal state space. We prove that, under certain conditions, the dimensionality of this manifold is at most the dimensionality of the action space plus one. This is the first result of its kind, linking the geometry of the state space to the dimensionality of the action space. We empirically corroborate this upper bound for four MuJoCo environments. We further demonstrate the applicability of our result by learning a policy in this low dimensional representation. To do so we introduce an algorit

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keytiwari2022on

Related papers

• Taming "data-hungry" Reinforcement Learning? Stability In Continuous State-action Spaces (2024)2.26
• Topological Foundations Of Reinforcement Learning (2024)0.00
• Beyond Distributions: Geometric Action Control For Continuous Reinforcement Learning (2025)0.00
• Sample Efficient Reinforcement Learning In Continuous State Spaces: A Perspective Beyond Linearity (2021)0.00
• Achieving Sample And Computational Efficient Reinforcement Learning By Action Space Reduction Via Grouping (2023)0.00
• How To Discretize Continuous State-action Spaces In Q-learning: A Symbolic Control Approach (2024)3.58
• Low-dimensional State And Action Representation Learning With MDP Homomorphism Metrics (2021)0.00
• Learning State Abstractions For Transfer In Continuous Control (2020)0.00