A Relational Intervention Approach For Unsupervised Dynamics Generalization In Model-based Reinforcement Learning
2022 · Jixian Guo, Mingming Gong, Dacheng Tao
Abstract
The generalization of model-based reinforcement learning (MBRL) methods to environments with unseen transition dynamics is an important yet challenging problem. Existing methods try to extract environment-specified information \(Z\) from past transition segments to make the dynamics prediction model generalizable to different dynamics. However, because environments are not labelled, the extracted information inevitably contains redundant information unrelated to the dynamics in transition segments and thus fails to maintain a crucial property of \(Z\): \(Z\) should be similar in the same environment and dissimilar in different ones. As a result, the learned dynamics prediction function will deviate from the true one, which undermines the generalization ability. To tackle this problem, we introduce an interventional prediction module to estimate the probability of two estimated \(\hat\{z\}_i, \hat\{z\}_j\) belonging to the same environment. Furthermore, by utilizing the \(Z\)'s invarian
Authors
(none)
Tags
Stats
Related papers
- Trajectory-wise Multiple Choice Learning For Dynamics Generalization In Reinforcement Learning (2020)6.77
- Prototypical Context-aware Dynamics Generalization For High-dimensional Model-based Reinforcement Learning (2022)0.00
- Model-invariant State Abstractions For Model-based Reinforcement Learning (2021)0.00
- Dynamics Generalization Via Information Bottleneck In Deep Reinforcement Learning (2020)0.00
- Dynamics Generalisation In Reinforcement Learning Via Adaptive Context-aware Policies (2023)2.26
- Planning With Exploration: Addressing Dynamics Bottleneck In Model-based Reinforcement Learning (2020)0.00
- Gradient Coupling: The Hidden Barrier To Generalization In Agentic Reinforcement Learning (2025)0.00
- Reward-consistent Dynamics Models Are Strongly Generalizable For Offline Reinforcement Learning (2023)0.00