Causalcomrl: Context-based Offline Meta-reinforcement Learning With Causal Representation

Abstract

Context-based offline meta-reinforcement learning (OMRL) methods have achieved appealing success by leveraging pre-collected offline datasets to develop task representations that guide policy learning. However, current context-based OMRL methods often introduce spurious correlations, where task components are incorrectly correlated due to confounders. These correlations can degrade policy performance when the confounders in the test task differ from those in the training task. To address this problem, we propose CausalCOMRL, a context-based OMRL method that integrates causal representation learning. This approach uncovers causal relationships among the task components and incorporates the causal relationships into task representations, enhancing the generalizability of RL agents. We further improve the distinction of task representations from different tasks by using mutual information optimization and contrastive learning. Utilizing these causal task representations, we employ SAC to

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyzhang2025causalcomrl

Related papers

• Provably Improved Context-based Offline Meta-rl With Attention And Contrastive Learning (2021)0.00
• Scrutinize What We Ignore: Reining In Task Representation Shift Of Context-based Offline Meta Reinforcement Learning (2024)0.00
• On Context Distribution Shift In Task Representation Learning For Offline Meta RL (2023)0.00
• Entropy Regularized Task Representation Learning For Offline Meta-reinforcement Learning (2024)0.00
• Causal Reinforcement Learning Using Observational And Interventional Data (2021)0.00
• FOCAL: Efficient Fully-offline Meta-reinforcement Learning Via Distance Metric Learning And Behavior Regularization (2020)0.00
• BECAUSE: Bilinear Causal Representation For Generalizable Offline Model-based Reinforcement Learning (2024)0.00
• Learning By Doing: An Online Causal Reinforcement Learning Framework With Causal-aware Policy (2024)1.56