Memory Based Trajectory-conditioned Policies For Learning From Sparse Rewards

Abstract

Reinforcement learning with sparse rewards is challenging because an agent can rarely obtain non-zero rewards and hence, gradient-based optimization of parameterized policies can be incremental and slow. Recent work demonstrated that using a memory buffer of previous successful trajectories can result in more effective policies. However, existing methods may overly exploit past successful experiences, which can encourage the agent to adopt sub-optimal and myopic behaviors. In this work, instead of focusing on good experiences with limited diversity, we propose to learn a trajectory-conditioned policy to follow and expand diverse past trajectories from a memory buffer. Our method allows the agent to reach diverse regions in the state space and improve upon the past trajectories to reach new states. We empirically show that our approach significantly outperforms count-based exploration methods (parametric approach) and self-imitation learning (parametric approach with non-parametric memo

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyguo2019memory

Related papers

• Learning Self-imitating Diverse Policies (2018)0.00
• Reward-conditioned Policies (2019)0.00
• Adamemento: Adaptive Memory-assisted Policy Optimization For Reinforcement Learning (2024)0.00
• Multi-objective Model-based Policy Search For Data-efficient Learning With Sparse Rewards (2018)0.00
• Discovering And Exploiting Sparse Rewards In A Learned Behavior Space (2021)0.00
• Intrinsic Reward Policy Optimization For Sparse-reward Environments (2026)0.00
• Near-future Policy Optimization (2026)0.00
• Long-term Visitation Value For Deep Exploration In Sparse Reward Reinforcement Learning (2020)7.24