Trajectory-wise Multiple Choice Learning For Dynamics Generalization In Reinforcement Learning
2020 · Younggyo Seo, Kimin Lee, Ignasi Clavera, et al.
Abstract
Model-based reinforcement learning (RL) has shown great potential in various control tasks in terms of both sample-efficiency and final performance. However, learning a generalizable dynamics model robust to changes in dynamics remains a challenge since the target transition dynamics follow a multi-modal distribution. In this paper, we present a new model-based RL algorithm, coined trajectory-wise multiple choice learning, that learns a multi-headed dynamics model for dynamics generalization. The main idea is updating the most accurate prediction head to specialize each head in certain environments with similar dynamics, i.e., clustering environments. Moreover, we incorporate context learning, which encodes dynamics-specific information from past experiences into the context latent vector, enabling the model to perform online adaptation to unseen environments. Finally, to utilize the specialized prediction heads more effectively, we propose an adaptive planning method, which selects th
Authors
(none)
Tags
Stats
Related papers
- Reward-consistent Dynamics Models Are Strongly Generalizable For Offline Reinforcement Learning (2023)0.00
- Prototypical Context-aware Dynamics Generalization For High-dimensional Model-based Reinforcement Learning (2022)0.00
- A Relational Intervention Approach For Unsupervised Dynamics Generalization In Model-based Reinforcement Learning (2022)0.00
- Learning Multimodal Transition Dynamics For Model-based Reinforcement Learning (2017)0.00
- Dynamics Generalisation In Reinforcement Learning Via Adaptive Context-aware Policies (2023)2.26
- Live In The Moment: Learning Dynamics Model Adapted To Evolving Policy (2022)0.00
- Planning With Exploration: Addressing Dynamics Bottleneck In Model-based Reinforcement Learning (2020)0.00
- Model-based Reinforcement Learning For Control Under Time-varying Dynamics (2026)0.00