CEM-RL: Combining Evolutionary And Gradient-based Methods For Policy Search

Abstract

Deep neuroevolution and deep reinforcement learning (deep RL) algorithms are two popular approaches to policy search. The former is widely applicable and rather stable, but suffers from low sample efficiency. By contrast, the latter is more sample efficient, but the most sample efficient variants are also rather unstable and highly sensitive to hyper-parameter setting. So far, these families of methods have mostly been compared as competing tools. However, an emerging approach consists in combining them so as to get the best of both worlds. Two previously existing combinations use either an ad hoc evolutionary algorithm or a goal exploration process together with the Deep Deterministic Policy Gradient (DDPG) algorithm, a sample efficient off-policy deep RL algorithm. In this paper, we propose a different combination scheme using the simple cross-entropy method (CEM) and Twin Delayed Deep Deterministic policy gradient (td3), another off-policy deep RL algorithm which improves over ddpg.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keypourchot2018cem

Related papers

• Combining Evolution And Deep Reinforcement Learning For Policy Search: A Survey (2022)12.25
• Evolution-guided Policy Gradient In Reinforcement Learning (2018)0.00
• An Efficient Asynchronous Method For Integrating Evolutionary And Gradient-based Policy Search (2020)0.00
• Collaborative Evolutionary Reinforcement Learning (2019)0.00
• Erl-re\(^2\): Efficient Evolutionary Reinforcement Learning With Shared State Representation And Individual Policy Representation (2022)0.00
• Supplementing Gradient-based Reinforcement Learning With Simple Evolutionary Ideas (2023)0.00
• Importance Mixing: Improving Sample Reuse In Evolutionary Policy Search Methods (2018)0.00
• Solving Deep Reinforcement Learning Tasks With Evolution Strategies And Linear Policy Networks (2024)0.00