Exploration In Approximate Hyper-state Space For Meta Reinforcement Learning

Abstract

To rapidly learn a new task, it is often essential for agents to explore efficiently -- especially when performance matters from the first timestep. One way to learn such behaviour is via meta-learning. Many existing methods however rely on dense rewards for meta-training, and can fail catastrophically if the rewards are sparse. Without a suitable reward signal, the need for exploration during meta-training is exacerbated. To address this, we propose HyperX, which uses novel reward bonuses for meta-training to explore in approximate hyper-state space (where hyper-states represent the environment state and the agent's task belief). We show empirically that HyperX meta-learns better task-exploration and adapts more successfully to new tasks than existing methods.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyzintgraf2020exploration

Related papers

• HMRL: Hyper-meta Learning For Sparse Reward Reinforcement Learning Problem (2020)0.00
• Hypothesis Network Planned Exploration For Rapid Meta-reinforcement Learning Adaptation (2023)0.00
• First-explore, Then Exploit: Meta-learning To Solve Hard Exploration-exploitation Trade-offs (2023)0.00
• Decoupling Exploration And Exploitation For Meta-reinforcement Learning Without Sacrifices (2020)0.00
• REMAX: Relational Representation For Multi-agent Exploration (2020)2.26
• Boosting Hierarchical Reinforcement Learning With Meta-learning For Complex Task Adaptation (2024)0.00
• Meta-learning To Explore Via Memory Density Feedback (2025)0.00
• MESA: Cooperative Meta-exploration In Multi-agent Learning Through Exploiting State-action Space Structure (2024)2.26