MOBODY: Model Based Off-dynamics Offline Reinforcement Learning

Abstract

We study off-dynamics offline reinforcement learning, where the goal is to learn a policy from offline source and limited target datasets with mismatched dynamics. Existing methods either penalize the reward or discard source transitions occurring in parts of the transition space with high dynamics shift. As a result, they optimize the policy using data from low-shift regions, limiting exploration of high-reward states in the target domain that do not fall within these regions. Consequently, such methods often fail when the dynamics shift is significant or the optimal trajectories lie outside the low-shift regions. To overcome this limitation, we propose MOBODY, a Model-Based Off-Dynamics Offline RL algorithm that optimizes a policy using learned target dynamics transitions to explore the target domain, rather than only being trained with the low dynamics-shift transitions. For the dynamics learning, built on the observation that achieving the same next state requires taking different

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyguo2025mobody

Related papers

• Model-based Offline Reinforcement Learning With Pessimism-modulated Dynamics Belief (2022)0.00
• Behavioral Priors And Dynamics Models: Improving Performance And Domain Transfer In Offline RL (2021)0.00
• Reward-consistent Dynamics Models Are Strongly Generalizable For Offline Reinforcement Learning (2023)0.00
• Autoregressive Dynamics Models For Offline Policy Evaluation And Optimization (2021)0.00
• Overcoming Model Bias For Robust Offline Deep Reinforcement Learning (2020)11.58
• DARA: Dynamics-aware Reward Augmentation In Offline Reinforcement Learning (2022)0.00
• Morel : Model-based Offline Reinforcement Learning (2020)0.00
• Mitigating Distribution Shift In Model-based Offline RL Via Shifts-aware Reward Learning (2024)0.00