Offline Meta-reinforcement Learning With Advantage Weighting

Abstract

This paper introduces the offline meta-reinforcement learning (offline meta-RL) problem setting and proposes an algorithm that performs well in this setting. Offline meta-RL is analogous to the widely successful supervised learning strategy of pre-training a model on a large batch of fixed, pre-collected data (possibly from various tasks) and fine-tuning the model to a new task with relatively little data. That is, in offline meta-RL, we meta-train on fixed, pre-collected data from several tasks in order to adapt to a new task with a very small amount (less than 5 trajectories) of data from the new task. By nature of being offline, algorithms for offline meta-RL can utilize the largest possible pool of training data available and eliminate potentially unsafe or costly data collection during meta-training. This setting inherits the challenges of offline RL, but it differs significantly because offline RL does not generally consider a) transfer to new tasks or b) limited data from the te

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keymitchell2020offline

Related papers

• Offline Meta-reinforcement Learning With Online Self-supervision (2021)0.00
• Provable Domain Adaptation For Offline Reinforcement Learning With Limited Samples (2024)0.00
• Offline Meta Learning Of Exploration (2020)0.00
• Leveraging Offline Data In Online Reinforcement Learning (2022)0.00
• Efficient Online Reinforcement Learning Fine-tuning Need Not Retain Offline Data (2024)0.00
• Provably Improved Context-based Offline Meta-rl With Attention And Contrastive Learning (2021)0.00
• FOCAL: Efficient Fully-offline Meta-reinforcement Learning Via Distance Metric Learning And Behavior Regularization (2020)0.00
• Statistically Efficient Advantage Learning For Offline Reinforcement Learning In Infinite Horizons (2022)0.00