Atradiff: Accelerating Online Reinforcement Learning With Imaginary Trajectories

Abstract

Training autonomous agents with sparse rewards is a long-standing problem in online reinforcement learning (RL), due to low data efficiency. Prior work overcomes this challenge by extracting useful knowledge from offline data, often accomplished through the learning of action distribution from offline data and utilizing the learned distribution to facilitate online RL. However, since the offline data are given and fixed, the extracted knowledge is inherently limited, making it difficult to generalize to new tasks. We propose a novel approach that leverages offline data to learn a generative diffusion model, coined as Adaptive Trajectory Diffuser (ATraDiff). This model generates synthetic trajectories, serving as a form of data augmentation and consequently enhancing the performance of online RL methods. The key strength of our diffuser lies in its adaptability, allowing it to effectively handle varying trajectory lengths and mitigate distribution shifts between online and offline data.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyyang2024atradiff

Related papers

• Bitrajdiff: Bidirectional Trajectory Generation With Diffusion Models For Offline Reinforcement Learning (2025)0.00
• Diffstitch: Boosting Offline Reinforcement Learning With Diffusion-based Trajectory Stitching (2024)0.00
• Contrastive Diffuser: Planning Towards High Return States Via Contrastive Learning (2024)0.00
• GTA: Generative Trajectory Augmentation With Guidance For Offline Reinforcement Learning (2024)6.62
• Using Offline Data To Speed Up Reinforcement Learning In Procedurally Generated Environments (2023)6.77
• Trajdeleter: Enabling Trajectory Forgetting In Offline Reinforcement Learning Agents (2024)0.00
• Stable Continual Reinforcement Learning Via Diffusion-based Trajectory Replay (2024)0.00
• Madiff: Offline Multi-agent Learning With Diffusion Models (2023)2.26