Synergizing Quality-diversity With Descriptor-conditioned Reinforcement Learning

Abstract

A hallmark of intelligence is the ability to exhibit a wide range of effective behaviors. Inspired by this principle, Quality-Diversity algorithms, such as MAP-Elites, are evolutionary methods designed to generate a set of diverse and high-fitness solutions. However, as a genetic algorithm, MAP-Elites relies on random mutations, which can become inefficient in high-dimensional search spaces, thus limiting its scalability to more complex domains, such as learning to control agents directly from high-dimensional inputs. To address this limitation, advanced methods like PGA-MAP-Elites and DCG-MAP-Elites have been developed, which combine actor-critic techniques from Reinforcement Learning with MAP-Elites, significantly enhancing the performance and efficiency of Quality-Diversity algorithms in complex, high-dimensional tasks. While these methods have successfully leveraged the trained critic to guide more effective mutations, the potential of the trained actor remains underutilized in imp

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyfaldor2023synergizing

Related papers

• The Quality-diversity Transformer: Generating Behavior-conditioned Trajectories With Decision Transformers (2023)6.77
• Diversity Policy Gradient For Sample Efficient Quality-diversity Optimization (2020)11.58
• Approximating Gradients For Differentiable Quality Diversity In Reinforcement Learning (2022)0.00
• Harnessing Distribution Ratio Estimators For Learning Agents With Quality And Diversity (2020)0.00
• Learning In Sparse Rewards Settings Through Quality-diversity Algorithms (2022)0.00
• Phasic Diversity Optimization For Population-based Reinforcement Learning (2024)0.00
• Effective Diversity In Population Based Reinforcement Learning (2020)0.00
• Selection-expansion: A Unifying Framework For Motion-planning And Diversity Search Algorithms (2021)0.00