Ancestral Reinforcement Learning: Unifying Zeroth-order Optimization And Genetic Algorithms For Reinforcement Learning

Abstract

Reinforcement Learning (RL) offers a fundamental framework for discovering optimal action strategies through interactions within unknown environments. Recent advancement have shown that the performance and applicability of RL can significantly be enhanced by exploiting a population of agents in various ways. Zeroth-Order Optimization (ZOO) leverages an agent population to estimate the gradient of the objective function, enabling robust policy refinement even in non-differentiable scenarios. As another application, Genetic Algorithms (GA) boosts the exploration of policy landscapes by mutational generation of policy diversity in an agent population and its refinement by selection. A natural question is whether we can have the best of two worlds that the agent population can have. In this work, we propose Ancestral Reinforcement Learning (ARL), which synergistically combines the robust gradient estimation of ZOO with the exploratory power of GA. The key idea in ARL is that each agent wit

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keynakashima2024ancestral

Related papers

• Synergizing Reinforcement Learning And Genetic Algorithms For Neural Combinatorial Optimization (2025)0.00
• Improving Generalization In Meta Reinforcement Learning Using Learned Objectives (2019)0.00
• Zeroth-order Optimization Is Secretly Single-step Policy Optimization (2025)0.00
• Supplementing Gradient-based Reinforcement Learning With Simple Evolutionary Ideas (2023)0.00
• A Unified Framework For Zero-shot Reinforcement Learning (2025)0.00
• Evolution-guided Policy Gradient In Reinforcement Learning (2018)0.00
• Evolutionary Reinforcement Learning: A Survey (2023)13.93
• Lineage Evolution Reinforcement Learning (2020)0.00