Powderworld: A Platform For Understanding Generalization Via Rich Task Distributions

Abstract

One of the grand challenges of reinforcement learning is the ability to generalize to new tasks. However, general agents require a set of rich, diverse tasks to train on. Designing a `foundation environment' for such tasks is tricky -- the ideal environment would support a range of emergent phenomena, an expressive task space, and fast runtime. To take a step towards addressing this research bottleneck, this work presents Powderworld, a lightweight yet expressive simulation environment running directly on the GPU. Within Powderworld, two motivating challenges distributions are presented, one for world-modelling and one for reinforcement learning. Each contains hand-designed test tasks to examine generalization. Experiments indicate that increasing the environment's complexity improves generalization for world models and certain reinforcement learning agents, yet may inhibit learning in high-variance environments. Powderworld aims to support the study of generalization by providing a so

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyfrans2022powderworld

Related papers

• Task Aware Dreamer For Task Generalization In Reinforcement Learning (2023)0.00
• Illuminating Generalization In Deep Reinforcement Learning Through Procedural Level Generation (2018)0.00
• Procedural Generalization By Planning With Self-supervised World Models (2021)0.00
• Assessing Generalization In Deep Reinforcement Learning (2018)0.00
• Quantifying The Effects Of Environment And Population Diversity In Multi-agent Reinforcement Learning (2021)9.03
• Improving Generalization In Meta-rl With Imaginary Tasks From Latent Dynamics Mixture (2021)0.00
• Quantifying Generalization In Reinforcement Learning (2018)0.00
• Zipfian Environments For Reinforcement Learning (2022)0.00