Accelerating Reinforcement Learning With A Directional-gaussian-smoothing Evolution Strategy

Abstract

Evolution strategy (ES) has been shown great promise in many challenging reinforcement learning (RL) tasks, rivaling other state-of-the-art deep RL methods. Yet, there are two limitations in the current ES practice that may hinder its otherwise further capabilities. First, most current methods rely on Monte Carlo type gradient estimators to suggest search direction, where the policy parameter is, in general, randomly sampled. Due to the low accuracy of such estimators, the RL training may suffer from slow convergence and require more iterations to reach optimal solution. Secondly, the landscape of reward functions can be deceptive and contains many local maxima, causing ES algorithms to prematurely converge and be unable to explore other parts of the parameter space with potentially greater rewards. In this work, we employ a Directional Gaussian Smoothing Evolutionary Strategy (DGS-ES) to accelerate RL training, which is well-suited to address these two challenges with its ability to i

Stats

• citations7
• S2 citations—
• github stars0
• HF likes0
• heat score6.77
• arxiv keyzhang2020accelerating

Related papers

• An Efficient Asynchronous Method For Integrating Evolutionary And Gradient-based Policy Search (2020)0.00
• Improving Exploration In Evolution Strategies For Deep Reinforcement Learning Via A Population Of Novelty-seeking Agents (2017)0.00
• Supplementing Gradient-based Reinforcement Learning With Simple Evolutionary Ideas (2023)0.00
• Evolution-guided Policy Gradient In Reinforcement Learning (2018)0.00
• Variance Reduction For Evolution Strategies Via Structured Control Variates (2019)0.00
• Improving Sample Efficiency In Evolutionary RL Using Off-policy Ranking (2022)0.00
• Deep Reinforcement Learning Versus Evolution Strategies: A Comparative Survey (2021)0.00
• Meta Reinforcement Learning With Distribution Of Exploration Parameters Learned By Evolution Strategies (2018)0.00