Decoupling Meta-reinforcement Learning With Gaussian Task Contexts And Skills

Abstract

Offline meta-reinforcement learning (meta-RL) methods, which adapt to unseen target tasks with prior experience, are essential in robot control tasks. Current methods typically utilize task contexts and skills as prior experience, where task contexts are related to the information within each task and skills represent a set of temporally extended actions for solving subtasks. However, these methods still suffer from limited performance when adapting to unseen target tasks, mainly because the learned prior experience lacks generalization, i.e., they are unable to extract effective prior experience from meta-training tasks by exploration and learning of continuous latent spaces. We propose a framework called decoupled meta-reinforcement learning (DCMRL), which (1) contrastively restricts the learning of task contexts through pulling in similar task contexts within the same task and pushing away different task contexts of different tasks, and (2) utilizes a Gaussian quantization variation

Stats

• citations1
• S2 citations—
• github stars0
• HF likes0
• heat score2.26
• arxiv keyhe2023decoupling

Related papers

• Efficient Off-policy Meta-reinforcement Learning Via Probabilistic Context Variables (2019)0.00
• On Context Distribution Shift In Task Representation Learning For Offline Meta RL (2023)0.00
• Entropy Regularized Task Representation Learning For Offline Meta-reinforcement Learning (2024)0.00
• Decoupling Exploration And Exploitation For Meta-reinforcement Learning Without Sacrifices (2020)0.00
• Skill-aware Mutual Information Optimisation For Generalisation In Reinforcement Learning (2024)0.00
• Improving Generalization In Meta-rl With Imaginary Tasks From Latent Dynamics Mixture (2021)0.00
• Scrutinize What We Ignore: Reining In Task Representation Shift Of Context-based Offline Meta Reinforcement Learning (2024)0.00
• Meta-reinforcement Learning Based On Self-supervised Task Representation Learning (2023)7.81