Information-theoretic Task Selection For Meta-reinforcement Learning

Abstract

In Meta-Reinforcement Learning (meta-RL) an agent is trained on a set of tasks to prepare for and learn faster in new, unseen, but related tasks. The training tasks are usually hand-crafted to be representative of the expected distribution of test tasks and hence all used in training. We show that given a set of training tasks, learning can be both faster and more effective (leading to better performance in the test tasks), if the training tasks are appropriately selected. We propose a task selection algorithm, Information-Theoretic Task Selection (ITTS), based on information theory, which optimizes the set of tasks used for training in meta-RL, irrespectively of how they are generated. The algorithm establishes which training tasks are both sufficiently relevant for the test tasks, and different enough from one another. We reproduce different meta-RL experiments from the literature and show that ITTS improves the final performance in all of them.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keygutierrez2020information

Related papers

• On The Effectiveness Of Fine-tuning Versus Meta-reinforcement Learning (2022)0.00
• A Tutorial On Meta-reinforcement Learning (2023)10.85
• Coreset-based Task Selection For Sample-efficient Meta-reinforcement Learning (2025)0.00
• Efficient Meta Reinforcement Learning For Preference-based Fast Adaptation (2022)0.00
• Model-based Adversarial Meta-reinforcement Learning (2020)0.00
• Unsupervised Meta-learning For Reinforcement Learning (2018)0.00
• Meta Reinforcement Learning With Finite Training Tasks -- A Density Estimation Approach (2022)0.00
• Procedural Generation Of Meta-reinforcement Learning Tasks (2023)0.00