Data-driven Evaluation Of Training Action Space For Reinforcement Learning

Abstract

Training action space selection for reinforcement learning (RL) is conflict-prone due to complex state-action relationships. To address this challenge, this paper proposes a Shapley-inspired methodology for training action space categorization and ranking. To reduce exponential-time shapley computations, the methodology includes a Monte Carlo simulation to avoid unnecessary explorations. The effectiveness of the methodology is illustrated using a cloud infrastructure resource tuning case study. It reduces the search space by 80% and categorizes the training action sets into dispensable and indispensable groups. Additionally, it ranks different training actions to facilitate high-performance yet cost-efficient RL model design. The proposed data-driven methodology is extensible to different domains, use cases, and reinforcement learning algorithms.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyghosh2022data

Related papers

• Achieving Sample And Computational Efficient Reinforcement Learning By Action Space Reduction Via Grouping (2023)0.00
• Reducing Action Space For Deep Reinforcement Learning Via Causal Effect Estimation (2025)0.00
• No Prior Mask: Eliminate Redundant Action For Deep Reinforcement Learning (2023)1.81
• Growing Action Spaces (2019)0.00
• Excluding The Irrelevant: Focusing Reinforcement Learning Through Continuous Action Masking (2024)4.52
• Extremum-seeking Action Selection For Accelerating Policy Optimization (2024)0.00
• Reinforcement Learning With Sparse-executing Actions Via Sparsity Regularization (2021)0.00
• Leveraging Factored Action Spaces For Efficient Offline Reinforcement Learning In Healthcare (2023)2.26