Dual-robust Cross-domain Offline Reinforcement Learning Against Dynamics Shifts
2025 · Zhongjian Qiao, Rui Yang, Jiafei Lyu, et al.
Abstract
Single-domain offline reinforcement learning (RL) often suffers from limited data coverage, while cross-domain offline RL handles this issue by leveraging additional data from other domains with dynamics shifts. However, existing studies primarily focus on train-time robustness (handling dynamics shifts from training data), neglecting the test-time robustness against dynamics perturbations when deployed in practical scenarios. In this paper, we investigate dual (both train-time and test-time) robustness against dynamics shifts in cross-domain offline RL. We first empirically show that the policy trained with cross-domain offline RL exhibits fragility under dynamics perturbations during evaluation, particularly when target domain data is limited. To address this, we introduce a novel robust cross-domain Bellman (RCB) operator, which enhances test-time robustness against dynamics perturbations while staying conservative to the out-of-distribution dynamics transitions, thus guaranteeing t
Authors
(none)
Tags
Stats
Related papers
- Efficient Cross-domain Offline Reinforcement Learning With Dynamics- And Value-aligned Data Filtering (2025)0.00
- Bridging Distributionally Robust Learning And Offline RL: An Approach To Mitigate Distribution Shift And Partial Data Coverage (2023)0.00
- Robust Adversarial Policy Optimization Under Dynamics Uncertainty (2026)0.00
- Beyond OOD State Actions: Supported Cross-domain Offline Reinforcement Learning (2023)0.00
- Behavioral Priors And Dynamics Models: Improving Performance And Domain Transfer In Offline RL (2021)0.00
- Domain Generalization For Robust Model-based Offline Reinforcement Learning (2022)0.00
- Towards Robust Policy: Enhancing Offline Reinforcement Learning With Adversarial Attacks And Defenses (2024)3.58
- Mitigating Distribution Shift In Model-based Offline RL Via Shifts-aware Reward Learning (2024)0.00