BECAUSE: Bilinear Causal Representation For Generalizable Offline Model-based Reinforcement Learning

Abstract

Offline model-based reinforcement learning (MBRL) enhances data efficiency by utilizing pre-collected datasets to learn models and policies, especially in scenarios where exploration is costly or infeasible. Nevertheless, its performance often suffers from the objective mismatch between model and policy learning, resulting in inferior performance despite accurate model predictions. This paper first identifies the primary source of this mismatch comes from the underlying confounders present in offline data for MBRL. Subsequently, we introduce \textbf\{B\}ilin\textbf\{E\}ar \textbf\{CAUS\}al r\textbf\{E\}presentation~(BECAUSE), an algorithm to capture causal representation for both states and actions to reduce the influence of the distribution shift, thus mitigating the objective mismatch problem. Comprehensive evaluations on 18 tasks that vary in data quality and environment context demonstrate the superior performance of BECAUSE over existing offline RL algorithms. We show the generali

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keylin2024because

Related papers

• Enhancing Offline Model-based RL Via Active Model Selection: A Bayesian Optimization Perspective (2025)0.00
• Behavioral Priors And Dynamics Models: Improving Performance And Domain Transfer In Offline RL (2021)0.00
• Morel : Model-based Offline Reinforcement Learning (2020)0.00
• Why Online Reinforcement Learning Is Causal (2024)0.00
• Bayes Adaptive Monte Carlo Tree Search For Offline Model-based Reinforcement Learning (2024)0.00
• Policy-driven World Model Adaptation For Robust Offline Model-based Reinforcement Learning (2025)0.00
• Behavior Prior Representation Learning For Offline Reinforcement Learning (2022)0.00
• Behavior Estimation From Multi-source Data For Offline Reinforcement Learning (2022)2.26