Derivative-free Reinforcement Learning: A Review

Abstract

Reinforcement learning is about learning agent models that make the best sequential decisions in unknown environments. In an unknown environment, the agent needs to explore the environment while exploiting the collected information, which usually forms a sophisticated problem to solve. Derivative-free optimization, meanwhile, is capable of solving sophisticated problems. It commonly uses a sampling-and-updating framework to iteratively improve the solution, where exploration and exploitation are also needed to be well balanced. Therefore, derivative-free optimization deals with a similar core issue as reinforcement learning, and has been introduced in reinforcement learning approaches, under the names of learning classifier systems and neuroevolution/evolutionary reinforcement learning. Although such methods have been developed for decades, recently, derivative-free reinforcement learning exhibits attracting increasing attention. However, recent survey on this topic is still lacking. I

Stats

• citations37
• S2 citations—
• github stars0
• HF likes0
• heat score11.85
• arxiv keyqian2021derivative

Related papers

• Differentiable Evolutionary Reinforcement Learning (2025)0.00
• Evolutionary Reinforcement Learning: A Survey (2023)13.93
• Learning To Optimize For Reinforcement Learning (2023)0.00
• On The Model-based Stochastic Value Gradient For Continuous Reinforcement Learning (2020)0.00
• Reward Models In Deep Reinforcement Learning: A Survey (2025)0.00
• Model-free Reinforcement Learning For Model-based Control: Towards Safe, Interpretable And Sample-efficient Agents (2025)0.00
• Control-oriented Model-based Reinforcement Learning With Implicit Differentiation (2021)5.84
• Learning Sampling Policy For Faster Derivative Free Optimization (2021)0.00