Beyond Distributions: Geometric Action Control For Continuous Reinforcement Learning

Abstract

Gaussian policies have dominated continuous control in deep reinforcement learning (RL), yet they suffer from a fundamental mismatch: their unbounded support requires ad-hoc squashing functions that distort the geometry of bounded action spaces. While von Mises-Fisher (vMF) distributions offer a theoretically grounded alternative on the sphere, their reliance on Bessel functions and rejection sampling hinders practical adoption. We propose \textbf\{Geometric Action Control (GAC)\}, a novel action generation paradigm that preserves the geometric benefits of spherical distributions while \textit\{simplifying computation\}. GAC decomposes action generation into a direction vector and a learnable concentration parameter, enabling efficient interpolation between deterministic actions and uniform spherical noise. This design reduces parameter count from \(2d\) to \(d+1\), and avoids the \(O(dk)\) complexity of vMF rejection sampling, achieving simple \(O(d)\) operations. Empirically, GAC con

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keylin2025beyond

Related papers

• Distributions As Actions: A Unified Framework For Diverse Action Spaces (2025)0.00
• Categorical Policies: Multimodal Policy Learning And Exploration In Continuous Control (2025)0.00
• Marginal Policy Gradients: A Unified Family Of Estimators For Bounded Action Spaces With Applications (2018)0.00
• Distributional Policy Optimization: An Alternative Approach For Continuous Control (2019)0.00
• Autoregressive Policies For Continuous Control Deep Reinforcement Learning (2019)7.50
• On The Geometry Of Reinforcement Learning In Continuous State And Action Spaces (2022)0.00
• Proximal Policy Optimization With Continuous Bounded Action Space Via The Beta Distribution (2021)0.00
• Action Gaps And Advantages In Continuous-time Distributional Reinforcement Learning (2024)0.00