Model-based Trajectory Stitching For Improved Offline Reinforcement Learning

Abstract

In many real-world applications, collecting large and high-quality datasets may be too costly or impractical. Offline reinforcement learning (RL) aims to infer an optimal decision-making policy from a fixed set of data. Getting the most information from historical data is then vital for good performance once the policy is deployed. We propose a model-based data augmentation strategy, Trajectory Stitching (TS), to improve the quality of sub-optimal historical trajectories. TS introduces unseen actions joining previously disconnected states: using a probabilistic notion of state reachability, it effectively `stitches' together parts of the historical demonstrations to generate new, higher quality ones. A stitching event consists of a transition between a pair of observed states through a synthetic and highly probable action. New actions are introduced only when they are expected to be beneficial, according to an estimated state-value function. We show that using this data augmentation st

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyhepburn2022model

Related papers

• Diffstitch: Boosting Offline Reinforcement Learning With Diffusion-based Trajectory Stitching (2024)0.00
• BATS: Best Action Trajectory Stitching (2022)0.00
• Offline RL With Observation Histories: Analyzing And Improving Sample Complexity (2023)0.00
• Offline Trajectory Optimization For Offline Reinforcement Learning (2024)1.20
• Harnessing Mixed Offline Reinforcement Learning Datasets Via Trajectory Weighting (2023)0.00
• GTA: Generative Trajectory Augmentation With Guidance For Offline Reinforcement Learning (2024)6.62
• Enhancing Offline Reinforcement Learning With Curriculum Learning-based Trajectory Valuation (2025)0.00
• Offline Safe Reinforcement Learning Using Trajectory Classification (2024)0.00