Contrastive Diffuser: Planning Towards High Return States Via Contrastive Learning

Abstract

The performance of offline reinforcement learning (RL) is sensitive to the proportion of high-return trajectories in the offline dataset. However, in many simulation environments and real-world scenarios, there are large ratios of low-return trajectories rather than high-return trajectories, which makes learning an efficient policy challenging. In this paper, we propose a method called Contrastive Diffuser (CDiffuser) to make full use of low-return trajectories and improve the performance of offline RL algorithms. Specifically, CDiffuser groups the states of trajectories in the offline dataset into high-return states and low-return states and treats them as positive and negative samples correspondingly. Then, it designs a contrastive mechanism to pull the trajectory of an agent toward high-return states and push them away from low-return states. Through the contrast mechanism, trajectories with low returns can serve as negative examples for policy learning, guiding the agent to avoid a

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyshan2024contrastive

Related papers

• Diffstitch: Boosting Offline Reinforcement Learning With Diffusion-based Trajectory Stitching (2024)0.00
• Atradiff: Accelerating Online Reinforcement Learning With Imaginary Trajectories (2024)0.00
• Preferred-action-optimized Diffusion Policies For Offline Reinforcement Learning (2024)0.00
• Diffusion Policies As An Expressive Policy Class For Offline Reinforcement Learning (2022)0.00
• Bitrajdiff: Bidirectional Trajectory Generation With Diffusion Models For Offline Reinforcement Learning (2025)0.00
• Diffusion Policies Creating A Trust Region For Offline Reinforcement Learning (2024)8.04
• Harnessing Mixed Offline Reinforcement Learning Datasets Via Trajectory Weighting (2023)0.00
• Diffpogan: Diffusion Policies With Generative Adversarial Networks For Offline Reinforcement Learning (2024)0.00