Introducing Symmetries To Black Box Meta Reinforcement Learning

Abstract

Meta reinforcement learning (RL) attempts to discover new RL algorithms automatically from environment interaction. In so-called black-box approaches, the policy and the learning algorithm are jointly represented by a single neural network. These methods are very flexible, but they tend to underperform in terms of generalisation to new, unseen environments. In this paper, we explore the role of symmetries in meta-generalisation. We show that a recent successful meta RL approach that meta-learns an objective for backpropagation-based learning exhibits certain symmetries (specifically the reuse of the learning rule, and invariance to input and output permutations) that are not present in typical black-box meta RL systems. We hypothesise that these symmetries can play an important role in meta-generalisation. Building off recent work in black-box supervised meta learning, we develop a black-box meta RL system that exhibits these same symmetries. We show through careful experimentation tha

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keykirsch2021introducing

Related papers

• Black Box Meta-learning Intrinsic Rewards (2024)0.00
• Learning From Symmetry: Meta-reinforcement Learning With Symmetrical Behaviors And Language Instructions (2022)0.00
• Improving Generalization In Meta Reinforcement Learning Using Learned Objectives (2019)0.00
• A Tutorial On Meta-reinforcement Learning (2023)10.85
• Learning To Reinforcement Learn (2016)0.00
• Theoretical Analysis Of Meta Reinforcement Learning: Generalization Bounds And Convergence Guarantees (2024)10.35
• RL\(^3\): Boosting Meta Reinforcement Learning Via RL Inside RL\(^2\) (2023)0.00
• Meta-gradient Reinforcement Learning With An Objective Discovered Online (2020)0.00