Solving Continual Offline RL Through Selective Weights Activation On Aligned Spaces

Abstract

Continual offline reinforcement learning (CORL) has shown impressive ability in diffusion-based lifelong learning systems by modeling the joint distributions of trajectories. However, most research only focuses on limited continual task settings where the tasks have the same observation and action space, which deviates from the realistic demands of training agents in various environments. In view of this, we propose Vector-Quantized Continual Diffuser, named VQ-CD, to break the barrier of different spaces between various tasks. Specifically, our method contains two complementary sections, where the quantization spaces alignment provides a unified basis for the selective weights activation. In the quantized spaces alignment, we leverage vector quantization to align the different state and action spaces of various tasks, facilitating continual training in the same space. Then, we propose to leverage a unified diffusion model attached by the inverse dynamic model to master all tasks by se

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyhu2024solving

Related papers

• DIAR: Diffusion-model-guided Implicit Q-learning With Adaptive Revaluation (2024)0.00
• Diffusion Policies Creating A Trust Region For Offline Reinforcement Learning (2024)8.04
• Diffusion Policies As An Expressive Policy Class For Offline Reinforcement Learning (2022)0.00
• Solving Continual Offline Reinforcement Learning With Decision Transformer (2024)0.00
• Preferred-action-optimized Diffusion Policies For Offline Reinforcement Learning (2024)0.00
• Contrastive Diffuser: Planning Towards High Return States Via Contrastive Learning (2024)0.00
• Diffusion-based Episodes Augmentation For Offline Multi-agent Reinforcement Learning (2024)0.00
• Boosting Continuous Control With Consistency Policy (2023)3.58