Model-based Adversarial Meta-reinforcement Learning

Abstract

Meta-reinforcement learning (meta-RL) aims to learn from multiple training tasks the ability to adapt efficiently to unseen test tasks. Despite the success, existing meta-RL algorithms are known to be sensitive to the task distribution shift. When the test task distribution is different from the training task distribution, the performance may degrade significantly. To address this issue, this paper proposes Model-based Adversarial Meta-Reinforcement Learning (AdMRL), where we aim to minimize the worst-case sub-optimality gap -- the difference between the optimal return and the return that the algorithm achieves after adaptation -- across all tasks in a family of tasks, with a model-based approach. We propose a minimax objective and optimize it by alternating between learning the dynamics model on a fixed task and finding the adversarial task for the current model -- the task for which the policy induced by the model is maximally suboptimal. Assuming the family of tasks is parameterized

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keylin2020model

Related papers

• Meta-reinforcement Learning With Universal Policy Adaptation: Provable Near-optimality Under All-task Optimum Comparator (2024)0.00
• Distributionally Adaptive Meta Reinforcement Learning (2022)2.26
• A Tutorial On Meta-reinforcement Learning (2023)10.85
• Boosting Exploration In Multi-task Reinforcement Learning Using Adversarial Networks (2022)0.00
• Meta-q-learning (2019)3.58
• Efficient Meta Reinforcement Learning For Preference-based Fast Adaptation (2022)0.00
• Context Meta-reinforcement Learning Via Neuromodulation (2021)6.34
• Decoupling Exploration And Exploitation For Meta-reinforcement Learning Without Sacrifices (2020)0.00