Reinforcement Learning With Sparse-executing Actions Via Sparsity Regularization

Abstract

Reinforcement learning (RL) has demonstrated impressive performance in decision-making tasks like embodied control, autonomous driving and financial trading. In many decision-making tasks, the agents often encounter the problem of executing actions under limited budgets. However, classic RL methods typically overlook the challenges posed by such sparse-executing actions. They operate under the assumption that all actions can be taken for a unlimited number of times, both in the formulation of the problem and in the development of effective algorithms. To tackle the issue of limited action execution in RL, this paper first formalizes the problem as a Sparse Action Markov Decision Process (SA-MDP), in which specific actions in the action space can only be executed for a limited time. Then, we propose a policy optimization algorithm, Action Sparsity REgularization (ASRE), which adaptively handles each action with a distinct preference. ASRE operates through two steps: First, ASRE evaluate

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keypang2021reinforcement

Related papers

• Sparsity Is Necessary: Polynomial-time Stability For Agentic Llms In Large Action Spaces (2026)0.00
• Learning Sparse Representations Incrementally In Deep Reinforcement Learning (2019)0.00
• A Regularized Approach To Sparse Optimal Policy In Reinforcement Learning (2019)0.00
• No Prior Mask: Eliminate Redundant Action For Deep Reinforcement Learning (2023)1.81
• ACERAC: Efficient Reinforcement Learning In Fine Time Discretization (2021)4.52
• Multi-objective Model-based Policy Search For Data-efficient Learning With Sparse Rewards (2018)0.00
• The Utility Of Sparse Representations For Control In Reinforcement Learning (2018)10.48
• Extremum-seeking Action Selection For Accelerating Policy Optimization (2024)0.00