GTA: Generative Trajectory Augmentation With Guidance For Offline Reinforcement Learning

Abstract

Offline Reinforcement Learning (Offline RL) presents challenges of learning effective decision-making policies from static datasets without any online interactions. Data augmentation techniques, such as noise injection and data synthesizing, aim to improve Q-function approximation by smoothing the learned state-action region. However, these methods often fall short of directly improving the quality of offline datasets, leading to suboptimal results. In response, we introduce GTA, Generative Trajectory Augmentation, a novel generative data augmentation approach designed to enrich offline data by augmenting trajectories to be both high-rewarding and dynamically plausible. GTA applies a diffusion model within the data augmentation framework. GTA partially noises original trajectories and then denoises them with classifier-free guidance via conditioning on amplified return value. Our results show that GTA, as a general data augmentation strategy, enhances the performance of widely used off

Stats

• citations2
• S2 citations—
• github stars32
• HF likes0
• heat score6.62
• arxiv keylee2024gta

Related papers

• Goal-conditioned Data Augmentation For Offline Reinforcement Learning (2024)0.00
• Offline Trajectory Optimization For Offline Reinforcement Learning (2024)1.20
• Model-based Trajectory Stitching For Improved Offline Reinforcement Learning (2022)0.00
• Using Offline Data To Speed Up Reinforcement Learning In Procedurally Generated Environments (2023)6.77
• Atradiff: Accelerating Online Reinforcement Learning With Imaginary Trajectories (2024)0.00
• Boosting Offline Reinforcement Learning With Residual Generative Modeling (2021)0.00
• Bitrajdiff: Bidirectional Trajectory Generation With Diffusion Models For Offline Reinforcement Learning (2025)0.00
• Equivariant Data Augmentation For Generalization In Offline Reinforcement Learning (2023)0.00