Emergent Complexity And Zero-shot Transfer Via Unsupervised Environment Design

Abstract

A wide range of reinforcement learning (RL) problems - including robustness, transfer learning, unsupervised RL, and emergent complexity - require specifying a distribution of tasks or environments in which a policy will be trained. However, creating a useful distribution of environments is error prone, and takes a significant amount of developer time and effort. We propose Unsupervised Environment Design (UED) as an alternative paradigm, where developers provide environments with unknown parameters, and these parameters are used to automatically produce a distribution over valid, solvable environments. Existing approaches to automatically generating environments suffer from common failure modes: domain randomization cannot generate structure or adapt the difficulty of the environment to the agent's learning progress, and minimax adversarial training leads to worst-case environments that are often unsolvable. To generate structured, solvable environments for our protagonist agent, we i

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keydennis2020emergent

Related papers

• DRED: Zero-shot Transfer In Reinforcement Learning Via Data-regularised Environment Design (2024)1.81
• Replay-guided Adversarial Environment Design (2021)0.00
• MAESTRO: Open-ended Environment Design For Multi-agent Reinforcement Learning (2023)0.00
• Learning To Design Games: Strategic Environments In Reinforcement Learning (2017)0.00
• Discovering General Reinforcement Learning Algorithms With Adversarial Environment Design (2023)0.00
• Generating Automatic Curricula Via Self-supervised Active Domain Randomization (2020)0.00
• One Solution Is Not All You Need: Few-shot Extrapolation Via Structured Maxent RL (2020)0.00
• How The Level Sampling Process Impacts Zero-shot Generalisation In Deep Reinforcement Learning (2023)0.00