Discovering Minimal Reinforcement Learning Environments

Abstract

Reinforcement learning (RL) agents are commonly trained and evaluated in the same environment. In contrast, humans often train in a specialized environment before being evaluated, such as studying a book before taking an exam. The potential of such specialized training environments is still vastly underexplored, despite their capacity to dramatically speed up training. The framework of synthetic environments takes a first step in this direction by meta-learning neural network-based Markov decision processes (MDPs). The initial approach was limited to toy problems and produced environments that did not transfer to unseen RL algorithms. We extend this approach in three ways: Firstly, we modify the meta-learning algorithm to discover environments invariant towards hyperparameter configurations and learning algorithms. Secondly, by leveraging hardware parallelism and introducing a curriculum on an agent's evaluation episode horizon, we can achieve competitive results on several challengi

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyliesen2024discovering

Related papers

• Learning Synthetic Environments And Reward Networks For Reinforcement Learning (2022)0.00
• Procedural Generation Of Meta-reinforcement Learning Tasks (2023)0.00
• Learning To Design Games: Strategic Environments In Reinforcement Learning (2017)0.00
• Discovering General Reinforcement Learning Algorithms With Adversarial Environment Design (2023)0.00
• Eden: A Unified Environment Framework For Booming Reinforcement Learning Algorithms (2021)0.00
• Towards A Domain-specific Modelling Environment For Reinforcement Learning (2024)0.00
• Learning To Reinforcement Learn (2016)0.00
• Learning Curricula In Open-ended Worlds (2023)0.00