Entropy Regularized Task Representation Learning For Offline Meta-reinforcement Learning

Abstract

Offline meta-reinforcement learning aims to equip agents with the ability to rapidly adapt to new tasks by training on data from a set of different tasks. Context-based approaches utilize a history of state-action-reward transitions -- referred to as the context -- to infer representations of the current task, and then condition the agent, i.e., the policy and value function, on the task representations. Intuitively, the better the task representations capture the underlying tasks, the better the agent can generalize to new tasks. Unfortunately, context-based approaches suffer from distribution mismatch, as the context in the offline data does not match the context at test time, limiting their ability to generalize to the test tasks. This leads to the task representations overfitting to the offline training data. Intuitively, the task representations should be independent of the behavior policy used to collect the offline data. To address this issue, we approximately minimize the mutua

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keynakhaei2024entropy

Related papers

• Robust Task Representations For Offline Meta-reinforcement Learning Via Contrastive Learning (2022)0.00
• On Context Distribution Shift In Task Representation Learning For Offline Meta RL (2023)0.00
• Scrutinize What We Ignore: Reining In Task Representation Shift Of Context-based Offline Meta Reinforcement Learning (2024)0.00
• FOCAL: Efficient Fully-offline Meta-reinforcement Learning Via Distance Metric Learning And Behavior Regularization (2020)0.00
• Offline Meta-reinforcement Learning With Online Self-supervision (2021)0.00
• Provably Improved Context-based Offline Meta-rl With Attention And Contrastive Learning (2021)0.00
• Offline Meta Learning Of Exploration (2020)0.00
• Offline Meta-reinforcement Learning With Advantage Weighting (2020)0.00