A Globally Convergent Evolutionary Strategy For Stochastic Constrained Optimization With Applications To Reinforcement Learning
2022 · Youssef Diouane, Aurelien Lucchi, Vihang Patil
Abstract
Evolutionary strategies have recently been shown to achieve competing levels of performance for complex optimization problems in reinforcement learning. In such problems, one often needs to optimize an objective function subject to a set of constraints, including for instance constraints on the entropy of a policy or to restrict the possible set of actions or states accessible to an agent. Convergence guarantees for evolutionary strategies to optimize stochastic constrained problems are however lacking in the literature. In this work, we address this problem by designing a novel optimization algorithm with a sufficient decrease mechanism that ensures convergence and that is based only on estimates of the functions. We demonstrate the applicability of this algorithm on two types of experiments: i) a control task for maximizing rewards and ii) maximizing rewards subject to a non-relaxable set of constraints.
Authors
(none)
Tags
Stats
Related papers
- Effects Of Different Optimization Formulations In Evolutionary Reinforcement Learning On Diverse Behavior Generation (2021)2.26
- Efficacy Of Modern Neuro-evolutionary Strategies For Continuous Control Optimization (2019)0.00
- Variance Reduction For Evolution Strategies Via Structured Control Variates (2019)0.00
- On The Convergence Of Policy Gradient Methods To Nash Equilibria In General Stochastic Games (2022)0.00
- Global Convergence Of Policy Gradient Methods In Reinforcement Learning, Games And Control (2023)0.00
- Joint Optimization Of Multi-objective Reinforcement Learning With Policy Gradient Based Algorithm (2021)6.34
- Solving Deep Reinforcement Learning Tasks With Evolution Strategies And Linear Policy Networks (2024)0.00
- Qualitative Differences Between Evolutionary Strategies And Reinforcement Learning Methods For Control Of Autonomous Agents (2022)0.00