Stein Variational Goal Generation For Adaptive Exploration In Multi-goal Reinforcement Learning

Abstract

In multi-goal Reinforcement Learning, an agent can share experience between related training tasks, resulting in better generalization for new tasks at test time. However, when the goal space has discontinuities and the reward is sparse, a majority of goals are difficult to reach. In this context, a curriculum over goals helps agents learn by adapting training tasks to their current capabilities. In this work we propose Stein Variational Goal Generation (SVGG), which samples goals of intermediate difficulty for the agent, by leveraging a learned predictive model of its goal reaching capabilities. The distribution of goals is modeled with particles that are attracted in areas of appropriate difficulty using Stein Variational Gradient Descent. We show that SVGG outperforms state-of-the-art multi-goal Reinforcement Learning methods in terms of success coverage in hard exploration problems, and demonstrate that it is endowed with a useful recovery property when the environment changes.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keycastanet2022stein

Related papers

• Variational Automatic Curriculum Learning For Sparse-reward Cooperative Multi-agent Problems (2021)0.00
• Maximum Entropy Gain Exploration For Long Horizon Multi-goal Reinforcement Learning (2020)0.00
• Variational Empowerment As Representation Learning For Goal-based Reinforcement Learning (2021)0.00
• Automatic Goal Generation For Reinforcement Learning Agents (2017)0.00
• Self-supervised Goal-reaching Results In Multi-agent Cooperation And Exploration (2025)0.00
• SVDE: Scalable Value-decomposition Exploration For Cooperative Multi-agent Reinforcement Learning (2023)0.00
• Explore-go: Leveraging Exploration For Generalisation In Deep Reinforcement Learning (2024)0.00
• Dense And Diverse Goal Coverage In Multi Goal Reinforcement Learning (2025)0.00