Solving Deep Reinforcement Learning Tasks With Evolution Strategies And Linear Policy Networks

Abstract

Although deep reinforcement learning methods can learn effective policies for challenging problems such as Atari games and robotics tasks, algorithms are complex, and training times are often long. This study investigates how Evolution Strategies perform compared to gradient-based deep reinforcement learning methods. We use Evolution Strategies to optimize the weights of a neural network via neuroevolution, performing direct policy search. We benchmark both deep policy networks and networks consisting of a single linear layer from observations to actions for three gradient-based methods, such as Proximal Policy Optimization. These methods are evaluated against three classical Evolution Strategies and Augmented Random Search, which all use linear policy networks. Our results reveal that Evolution Strategies can find effective linear policies for many reinforcement learning benchmark tasks, unlike deep reinforcement learning methods that can only find successful policies using much large

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keywong2024solving

Related papers

• Evolution-guided Policy Gradient In Reinforcement Learning (2018)0.00
• Comparing Deep Reinforcement Learning And Evolutionary Methods In Continuous Control (2017)0.00
• Novelty Search For Deep Reinforcement Learning Policy Network Weights By Action Sequence Edit Metric Distance (2019)8.09
• Combining Evolution And Deep Reinforcement Learning For Policy Search: A Survey (2022)12.25
• An Efficient Asynchronous Method For Integrating Evolutionary And Gradient-based Policy Search (2020)0.00
• CEM-RL: Combining Evolutionary And Gradient-based Methods For Policy Search (2018)0.00
• Qualitative Differences Between Evolutionary Strategies And Reinforcement Learning Methods For Control Of Autonomous Agents (2022)0.00
• Improving Exploration In Evolution Strategies For Deep Reinforcement Learning Via A Population Of Novelty-seeking Agents (2017)0.00