A Tutorial On Meta-reinforcement Learning

Abstract

While deep reinforcement learning (RL) has fueled multiple high-profile successes in machine learning, it is held back from more widespread adoption by its often poor data efficiency and the limited generality of the policies it produces. A promising approach for alleviating these limitations is to cast the development of better RL algorithms as a machine learning problem itself in a process called meta-RL. Meta-RL is most commonly studied in a problem setting where, given a distribution of tasks, the goal is to learn a policy that is capable of adapting to any new task from the task distribution with as little data as possible. In this survey, we describe the meta-RL problem setting in detail as well as its major variations. We discuss how, at a high level, meta-RL research can be clustered based on the presence of a task distribution and the learning budget available for each individual task. Using these clusters, we then survey meta-RL algorithms and applications. We conclude by pre

Stats

• citations27
• S2 citations—
• github stars0
• HF likes0
• heat score10.85
• arxiv keybeck2023a

Related papers

• Learning To Reinforcement Learn (2016)0.00
• Guided Meta-policy Search (2019)0.00
• RL\(^3\): Boosting Meta Reinforcement Learning Via RL Inside RL\(^2\) (2023)0.00
• Distributionally Adaptive Meta Reinforcement Learning (2022)2.26
• Model-based Adversarial Meta-reinforcement Learning (2020)0.00
• Improving Generalization In Meta Reinforcement Learning Using Learned Objectives (2019)0.00
• Unsupervised Meta-learning For Reinforcement Learning (2018)0.00
• On The Effectiveness Of Fine-tuning Versus Meta-reinforcement Learning (2022)0.00