Harmonydream: Task Harmonization Inside World Models

Abstract

Model-based reinforcement learning (MBRL) holds the promise of sample-efficient learning by utilizing a world model, which models how the environment works and typically encompasses components for two tasks: observation modeling and reward modeling. In this paper, through a dedicated empirical investigation, we gain a deeper understanding of the role each task plays in world models and uncover the overlooked potential of sample-efficient MBRL by mitigating the domination of either observation or reward modeling. Our key insight is that while prevalent approaches of explicit MBRL attempt to restore abundant details of the environment via observation models, it is difficult due to the environment's complexity and limited model capacity. On the other hand, reward models, while dominating implicit MBRL and adept at learning compact task-centric dynamics, are inadequate for sample-efficient learning without richer learning signals. Motivated by these insights and discoveries, we propose a s

Stats

• citations0
• S2 citations—
• github stars53
• HF likes0
• heat score3.46
• arxiv keyma2023harmonydream

Related papers

• A Model-based Approach For Sample-efficient Multi-task Reinforcement Learning (2019)0.00
• Exploring The Limits Of Hierarchical World Models In Reinforcement Learning (2024)6.34
• PWM: Policy Learning With Multi-task World Models (2024)0.00
• Dreamsmooth: Improving Model-based Reinforcement Learning Via Reward Smoothing (2023)0.00
• Task Aware Dreamer For Task Generalization In Reinforcement Learning (2023)0.00
• Bridging Imagination And Reality For Model-based Deep Reinforcement Learning (2020)0.00
• MABL: Bi-level Latent-variable World Model For Sample-efficient Multi-agent Reinforcement Learning (2023)0.00
• Synthesizing World Models For Bilevel Planning (2025)0.00