Maximum Entropy Gain Exploration For Long Horizon Multi-goal Reinforcement Learning

Abstract

What goals should a multi-goal reinforcement learning agent pursue during training in long-horizon tasks? When the desired (test time) goal distribution is too distant to offer a useful learning signal, we argue that the agent should not pursue unobtainable goals. Instead, it should set its own intrinsic goals that maximize the entropy of the historical achieved goal distribution. We propose to optimize this objective by having the agent pursue past achieved goals in sparsely explored areas of the goal space, which focuses exploration on the frontier of the achievable goal set. We show that our strategy achieves an order of magnitude better sample efficiency than the prior state of the art on long-horizon multi-goal tasks including maze navigation and block stacking.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keypitis2020maximum

Related papers

• Maximum Entropy-regularized Multi-goal Reinforcement Learning (2019)0.00
• Task-agnostic Exploration Via Policy Gradient Of A Non-parametric State Entropy Estimate (2020)0.00
• Fast Rates For Maximum Entropy Exploration (2023)0.00
• Dense And Diverse Goal Coverage In Multi Goal Reinforcement Learning (2025)0.00
• Self-supervised Goal-reaching Results In Multi-agent Cooperation And Exploration (2025)0.00
• Provably Efficient Maximum Entropy Exploration (2018)0.00
• Maximum-entropy Exploration With Future State-action Visitation Measures (2026)0.00
• Maximum Entropy Exploration Without The Rollouts (2026)0.00