Meta Reinforcement Learning With Finite Training Tasks -- A Density Estimation Approach

Abstract

In meta reinforcement learning (meta RL), an agent learns from a set of training tasks how to quickly solve a new task, drawn from the same task distribution. The optimal meta RL policy, a.k.a. the Bayes-optimal behavior, is well defined, and guarantees optimal reward in expectation, taken with respect to the task distribution. The question we explore in this work is how many training tasks are required to guarantee approximately optimal behavior with high probability. Recent work provided the first such PAC analysis for a model-free setting, where a history-dependent policy was learned from the training tasks. In this work, we propose a different approach: directly learn the task distribution, using density estimation techniques, and then train a policy on the learned task distribution. We show that our approach leads to bounds that depend on the dimension of the task distribution. In particular, in settings where the task distribution lies in a low-dimensional manifold, we extend our

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyrimon2022meta

Related papers

• A Tutorial On Meta-reinforcement Learning (2023)10.85
• Distributionally Adaptive Meta Reinforcement Learning (2022)2.26
• Improving Generalization In Meta-rl With Imaginary Tasks From Latent Dynamics Mixture (2021)0.00
• Guided Meta-policy Search (2019)0.00
• Model-based Adversarial Meta-reinforcement Learning (2020)0.00
• Information-theoretic Task Selection For Meta-reinforcement Learning (2020)0.00
• Procedural Generation Of Meta-reinforcement Learning Tasks (2023)0.00
• Efficient Meta Reinforcement Learning For Preference-based Fast Adaptation (2022)0.00