Excluding The Irrelevant: Focusing Reinforcement Learning Through Continuous Action Masking

Abstract

Continuous action spaces in reinforcement learning (RL) are commonly defined as multidimensional intervals. While intervals usually reflect the action boundaries for tasks well, they can be challenging for learning because the typically large global action space leads to frequent exploration of irrelevant actions. Yet, little task knowledge can be sufficient to identify significantly smaller state-specific sets of relevant actions. Focusing learning on these relevant actions can significantly improve training efficiency and effectiveness. In this paper, we propose to focus learning on the set of relevant actions and introduce three continuous action masking methods for exactly mapping the action space to the state-dependent set of relevant actions. Thus, our methods ensure that only relevant actions are executed, enhancing the predictability of the RL agent and enabling its use in safety-critical applications. We further derive the implications of the proposed methods on the policy gra

Stats

• citations3
• S2 citations—
• github stars0
• HF likes0
• heat score4.52
• arxiv keystolz2024excluding

Related papers

• No Prior Mask: Eliminate Redundant Action For Deep Reinforcement Learning (2023)1.81
• CAMEL: Continuous Action Masking Enabled By Large Language Models For Reinforcement Learning (2025)0.00
• Action Mapping For Reinforcement Learning In Continuous Environments With Constraints (2024)0.00
• Active Inference And Reinforcement Learning: A Unified Inference On Continuous State And Action Spaces Under Partial Observability (2022)5.84
• Lifelong Reinforcement Learning With Modulating Masks (2022)0.00
• Taming "data-hungry" Reinforcement Learning? Stability In Continuous State-action Spaces (2024)2.26
• Sample Efficient Reinforcement Learning In Continuous State Spaces: A Perspective Beyond Linearity (2021)0.00
• Deep RL With Information Constrained Policies: Generalization In Continuous Control (2020)0.00