Reusability And Transferability Of Macro Actions For Reinforcement Learning

Abstract

Conventional reinforcement learning (RL) typically determines an appropriate primitive action at each timestep. However, by using a proper macro action, defined as a sequence of primitive actions, an agent is able to bypass intermediate states to a farther state and facilitate its learning procedure. The problem we would like to investigate is what associated beneficial properties that macro actions may possess. In this paper, we unveil the properties of reusability and transferability of macro actions. The first property, reusability, means that a macro action generated along with one RL method can be reused by another RL method for training, while the second one, transferability, means that a macro action can be utilized for training agents in similar environments with different reward settings. In our experiments, we first generate macro actions along with RL methods. We then provide a set of analyses to reveal the properties of reusability and transferability of the generated macro

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keychang2019reusability

Related papers

• Hierarchical Meta-reinforcement Learning Via Automated Macro-action Discovery (2024)0.00
• Macro-action-based Deep Multi-agent Reinforcement Learning (2020)0.00
• Macro-action-based Multi-agent/robot Deep Reinforcement Learning Under Partial Observability (2022)5.84
• A Compression-inspired Framework For Macro Discovery (2017)0.00
• Contextual Pre-planning On Reward Machine Abstractions For Enhanced Transfer In Deep Reinforcement Learning (2023)2.26
• Macro Action Selection With Deep Reinforcement Learning In Starcraft (2018)9.92
• No Prior Mask: Eliminate Redundant Action For Deep Reinforcement Learning (2023)1.81
• Himac: Hierarchical Macro-micro Learning For Long-horizon LLM Agents (2026)0.00