An Efficient Asynchronous Method For Integrating Evolutionary And Gradient-based Policy Search

Abstract

Deep reinforcement learning (DRL) algorithms and evolution strategies (ES) have been applied to various tasks, showing excellent performances. These have the opposite properties, with DRL having good sample efficiency and poor stability, while ES being vice versa. Recently, there have been attempts to combine these algorithms, but these methods fully rely on synchronous update scheme, making it not ideal to maximize the benefits of the parallelism in ES. To solve this challenge, asynchronous update scheme was introduced, which is capable of good time-efficiency and diverse policy exploration. In this paper, we introduce an Asynchronous Evolution Strategy-Reinforcement Learning (AES-RL) that maximizes the parallel efficiency of ES and integrates it with policy gradient methods. Specifically, we propose 1) a novel framework to merge ES and DRL asynchronously and 2) various asynchronous update methods that can take all advantages of asynchronism, ES, and DRL, which are exploration and tim

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keylee2020an

Related papers

• Accelerating Reinforcement Learning With A Directional-gaussian-smoothing Evolution Strategy (2020)6.77
• Evolution-guided Policy Gradient In Reinforcement Learning (2018)0.00
• CEM-RL: Combining Evolutionary And Gradient-based Methods For Policy Search (2018)0.00
• Improving Exploration In Evolution Strategies For Deep Reinforcement Learning Via A Population Of Novelty-seeking Agents (2017)0.00
• Deep Reinforcement Learning Versus Evolution Strategies: A Comparative Survey (2021)0.00
• Erl-re\(^2\): Efficient Evolutionary Reinforcement Learning With Shared State Representation And Individual Policy Representation (2022)0.00
• Combining Evolution And Deep Reinforcement Learning For Policy Search: A Survey (2022)12.25
• Improving Sample Efficiency In Evolutionary RL Using Off-policy Ranking (2022)0.00