Exploring More When It Needs In Deep Reinforcement Learning

Abstract

We propose a exploration mechanism of policy in Deep Reinforcement Learning, which is exploring more when agent needs, called Add Noise to Noise (AN2N). The core idea is: when the Deep Reinforcement Learning agent is in a state of poor performance in history, it needs to explore more. So we use cumulative rewards to evaluate which past states the agents have not performed well, and use cosine distance to measure whether the current state needs to be explored more. This method shows that the exploration mechanism of the agent's policy is conducive to efficient exploration. We combining the proposed exploration mechanism AN2N with Deep Deterministic Policy Gradient (DDPG), Soft Actor-Critic (SAC) algorithms, and apply it to the field of continuous control tasks, such as halfCheetah, Hopper, and Swimmer, achieving considerable improvement in performance and convergence speed.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyguo2021exploring

Related papers

• Action Noise In Off-policy Deep Reinforcement Learning: Impact On Exploration And Performance (2022)0.00
• Parameter Space Noise For Exploration (2017)0.00
• Policy Augmentation: An Exploration Strategy For Faster Convergence Of Deep Reinforcement Learning Algorithms (2021)2.26
• Noisy Networks For Exploration (2017)0.00
• Improving Policy Gradient By Exploring Under-appreciated Rewards (2016)0.00
• Learning To Explore With Meta-policy Gradient (2018)0.00
• Guided Exploration In Reinforcement Learning Via Monte Carlo Critic Optimization (2022)0.00
• Nadpex: An On-policy Temporally Consistent Exploration Method For Deep Reinforcement Learning (2018)0.00