Learning Action Translator For Meta Reinforcement Learning On Sparse-reward Tasks

Abstract

Meta reinforcement learning (meta-RL) aims to learn a policy solving a set of training tasks simultaneously and quickly adapting to new tasks. It requires massive amounts of data drawn from training tasks to infer the common structure shared among tasks. Without heavy reward engineering, the sparse rewards in long-horizon tasks exacerbate the problem of sample efficiency in meta-RL. Another challenge in meta-RL is the discrepancy of difficulty level among tasks, which might cause one easy task dominating learning of the shared policy and thus preclude policy adaptation to new tasks. This work introduces a novel objective function to learn an action translator among training tasks. We theoretically verify that the value of the transferred policy with the action translator can be close to the value of the source policy and our objective function (approximately) upper bounds the value difference. We propose to combine the action translator with context-based meta-RL algorithms for better

Stats

• citations3
• S2 citations—
• github stars0
• HF likes0
• heat score4.52
• arxiv keyguo2022learning

Related papers

• HMRL: Hyper-meta Learning For Sparse Reward Reinforcement Learning Problem (2020)0.00
• Model-based Adversarial Meta-reinforcement Learning (2020)0.00
• A Tutorial On Meta-reinforcement Learning (2023)10.85
• Decoupling Exploration And Exploitation For Meta-reinforcement Learning Without Sacrifices (2020)0.00
• Hierarchical Meta-reinforcement Learning Via Automated Macro-action Discovery (2024)0.00
• Guided Meta-policy Search (2019)0.00
• Efficient Meta Reinforcement Learning For Preference-based Fast Adaptation (2022)0.00
• Context Meta-reinforcement Learning Via Neuromodulation (2021)6.34