Achieving Sample And Computational Efficient Reinforcement Learning By Action Space Reduction Via Grouping

Abstract

Reinforcement learning often needs to deal with the exponential growth of states and actions when exploring optimal control in high-dimensional spaces (often known as the curse of dimensionality). In this work, we address this issue by learning the inherent structure of action-wise similar MDP to appropriately balance the performance degradation versus sample/computational complexity. In particular, we partition the action spaces into multiple groups based on the similarity in transition distribution and reward function, and build a linear decomposition model to capture the difference between the intra-group transition kernel and the intra-group rewards. Both our theoretical analysis and experiments reveal a *surprising and counter-intuitive result*: while a more refined grouping strategy can reduce the approximation error caused by treating actions in the same group as identical, it also leads to increased estimation error when the size of samples or the computation resources is limit

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyli2023achieving

Related papers

• No Prior Mask: Eliminate Redundant Action For Deep Reinforcement Learning (2023)1.81
• Reducing Action Space For Deep Reinforcement Learning Via Causal Effect Estimation (2025)0.00
• Data-driven Evaluation Of Training Action Space For Reinforcement Learning (2022)0.00
• Sample-efficient Reinforcement Learning For Linearly-parameterized Mdps With A Generative Model (2021)0.00
• Low-dimensional State And Action Representation Learning With MDP Homomorphism Metrics (2021)0.00
• Q-function Decomposition With Intervention Semantics With Factored Action Spaces (2025)0.00
• On The Geometry Of Reinforcement Learning In Continuous State And Action Spaces (2022)0.00
• On The Sample Complexity Of Reinforcement Learning With Policy Space Generalization (2020)0.00