Explore-go: Leveraging Exploration For Generalisation In Deep Reinforcement Learning

Abstract

One of the remaining challenges in reinforcement learning is to develop agents that can generalise to novel scenarios they might encounter once deployed. This challenge is often framed in a multi-task setting where agents train on a fixed set of tasks and have to generalise to new tasks. Recent work has shown that in this setting increased exploration during training can be leveraged to increase the generalisation performance of the agent. This makes sense when the states encountered during testing can actually be explored during training. In this paper, we provide intuition why exploration can also benefit generalisation to states that cannot be explicitly encountered during training. Additionally, we propose a novel method Explore-Go that exploits this intuition by increasing the number of states on which the agent trains. Explore-Go effectively increases the starting state distribution of the agent and as a result can be used in conjunction with most existing on-policy or off-policy

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyweltevrede2024explore

Related papers

• Exploration Implies Data Augmentation: Reachability And Generalisation In Contextual Mdps (2024)0.00
• Go-explore: A New Approach For Hard-exploration Problems (2019)0.00
• Generative Adversarial Exploration For Reinforcement Learning (2022)0.00
• First Go, Then Post-explore: The Benefits Of Post-exploration In Intrinsic Motivation (2022)0.00
• Time-myopic Go-explore: Learning A State Representation For The Go-explore Paradigm (2023)0.00
• Improving Policy Gradient By Exploring Under-appreciated Rewards (2016)0.00
• A Survey Analyzing Generalization In Deep Reinforcement Learning (2024)0.00
• Off-policy Reinforcement Learning With Model-based Exploration Augmentation (2025)0.00