Enhancing Decision Transformer With Diffusion-based Trajectory Branch Generation
2024 · Zhihong Liu, Long Qian, Zeyang Liu, et al.
Abstract
Decision Transformer (DT) can learn effective policy from offline datasets by converting the offline reinforcement learning (RL) into a supervised sequence modeling task, where the trajectory elements are generated auto-regressively conditioned on the return-to-go (RTG).However, the sequence modeling learning approach tends to learn policies that converge on the sub-optimal trajectories within the dataset, for lack of bridging data to move to better trajectories, even if the condition is set to the highest RTG.To address this issue, we introduce Diffusion-Based Trajectory Branch Generation (BG), which expands the trajectories of the dataset with branches generated by a diffusion model.The trajectory branch is generated based on the segment of the trajectory within the dataset, and leads to trajectories with higher returns.We concatenate the generated branch with the trajectory segment as an expansion of the trajectory.After expanding, DT has more opportunities to learn policies to move
Authors
(none)
Tags
Stats
Related papers
- Bitrajdiff: Bidirectional Trajectory Generation With Diffusion Models For Offline Reinforcement Learning (2025)0.00
- Q-learning Decision Transformer: Leveraging Dynamic Programming For Conditional Sequence Modelling In Offline RL (2022)0.00
- Generalized Decision Transformer For Offline Hindsight Information Matching (2021)0.00
- DODT: Enhanced Online Decision Transformer Learning Through Dreamer's Actor-critic Trajectory Forecasting (2024)0.00
- Long-horizon Rollout Via Dynamics Diffusion For Offline Reinforcement Learning (2024)1.81
- Return Augmented Decision Transformer For Off-dynamics Reinforcement Learning (2024)0.00
- GTA: Generative Trajectory Augmentation With Guidance For Offline Reinforcement Learning (2024)6.62
- Switch Trajectory Transformer With Distributional Value Approximation For Multi-task Reinforcement Learning (2022)0.00