Provably Improved Context-based Offline Meta-rl With Attention And Contrastive Learning

Abstract

Meta-learning for offline reinforcement learning (OMRL) is an understudied problem with tremendous potential impact by enabling RL algorithms in many real-world applications. A popular solution to the problem is to infer task identity as augmented state using a context-based encoder, for which efficient learning of robust task representations remains an open challenge. In this work, we provably improve upon one of the SOTA OMRL algorithms, FOCAL, by incorporating intra-task attention mechanism and inter-task contrastive learning objectives, to robustify task representation learning against sparse reward and distribution shift. Theoretical analysis and experiments are presented to demonstrate the superior performance and robustness of our end-to-end and model-free framework compared to prior algorithms across multiple meta-RL benchmarks.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyli2021provably

Related papers

• Scrutinize What We Ignore: Reining In Task Representation Shift Of Context-based Offline Meta Reinforcement Learning (2024)0.00
• On Context Distribution Shift In Task Representation Learning For Offline Meta RL (2023)0.00
• FOCAL: Efficient Fully-offline Meta-reinforcement Learning Via Distance Metric Learning And Behavior Regularization (2020)0.00
• Causalcomrl: Context-based Offline Meta-reinforcement Learning With Causal Representation (2025)0.00
• Robust Task Representations For Offline Meta-reinforcement Learning Via Contrastive Learning (2022)0.00
• Entropy Regularized Task Representation Learning For Offline Meta-reinforcement Learning (2024)0.00
• Offline Meta-reinforcement Learning With Advantage Weighting (2020)0.00
• Enhancing Online Reinforcement Learning With Meta-learned Objective From Offline Data (2025)0.00