Qualitative Differences Between Evolutionary Strategies And Reinforcement Learning Methods For Control Of Autonomous Agents

Abstract

In this paper we analyze the qualitative differences between evolutionary strategies and reinforcement learning algorithms by focusing on two popular state-of-the-art algorithms: the OpenAI-ES evolutionary strategy and the Proximal Policy Optimization (PPO) reinforcement learning algorithm -- the most similar methods of the two families. We analyze how the methods differ with respect to: (i) general efficacy, (ii) ability to cope with sparse rewards, (iii) propensity/capacity to discover minimal solutions, (iv) dependency on reward shaping, and (v) ability to cope with variations of the environmental conditions. The analysis of the performance and of the behavioral strategies displayed by the agents trained with the two methods on benchmark problems enable us to demonstrate qualitative differences which were not identified in previous studies, to identify the relative weakness of the two methods, and to propose ways to ameliorate some of those weakness. We show that the characteristics

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keymilano2022qualitative

Related papers

• Comparing Deep Reinforcement Learning And Evolutionary Methods In Continuous Control (2017)0.00
• Efficacy Of Modern Neuro-evolutionary Strategies For Continuous Control Optimization (2019)0.00
• Deep Reinforcement Learning Versus Evolution Strategies: A Comparative Survey (2021)0.00
• Effects Of Different Optimization Formulations In Evolutionary Reinforcement Learning On Diverse Behavior Generation (2021)2.26
• Solving Deep Reinforcement Learning Tasks With Evolution Strategies And Linear Policy Networks (2024)0.00
• Proximal Policy Optimization Algorithms (2017)0.00
• Policy Optimization With Model-based Explorations (2018)5.84
• Performance Comparisons Of Reinforcement Learning Algorithms For Sequential Experimental Design (2025)0.00