Prototypical Context-aware Dynamics Generalization For High-dimensional Model-based Reinforcement Learning

Abstract

The latent world model provides a promising way to learn policies in a compact latent space for tasks with high-dimensional observations, however, its generalization across diverse environments with unseen dynamics remains challenging. Although the recurrent structure utilized in current advances helps to capture local dynamics, modeling only state transitions without an explicit understanding of environmental context limits the generalization ability of the dynamics model. To address this issue, we propose a Prototypical Context-Aware Dynamics (ProtoCAD) model, which captures the local dynamics by time consistent latent context and enables dynamics generalization in high-dimensional control tasks. ProtoCAD extracts useful contextual information with the help of the prototypes clustered over batch and benefits model-based RL in two folds: 1) It utilizes a temporally consistent prototypical regularizer that encourages the prototype assignments produced for different time parts of the sa

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keywang2022prototypical

Related papers

• Dynamics Generalisation In Reinforcement Learning Via Adaptive Context-aware Policies (2023)2.26
• Trajectory-wise Multiple Choice Learning For Dynamics Generalization In Reinforcement Learning (2020)6.77
• A Relational Intervention Approach For Unsupervised Dynamics Generalization In Model-based Reinforcement Learning (2022)0.00
• Decomposed Mutual Information Optimization For Generalized Context In Meta-reinforcement Learning (2022)0.00
• Reinforcement Learning Under Latent Dynamics: Toward Statistical And Algorithmic Modularity (2024)0.00
• Dynamics-adaptive Continual Reinforcement Learning Via Progressive Contextualization (2022)7.16
• Dynamic Sparsity: Challenging Common Sparsity Assumptions For Learning World Models In Robotic Reinforcement Learning Benchmarks (2025)0.00
• Mocoda: Model-based Counterfactual Data Augmentation (2022)2.26