Generative Adversarial Exploration For Reinforcement Learning

Abstract

Exploration is crucial for training the optimal reinforcement learning (RL) policy, where the key is to discriminate whether a state visiting is novel. Most previous work focuses on designing heuristic rules or distance metrics to check whether a state is novel without considering such a discrimination process that can be learned. In this paper, we propose a novel method called generative adversarial exploration (GAEX) to encourage exploration in RL via introducing an intrinsic reward output from a generative adversarial network, where the generator provides fake samples of states that help discriminator identify those less frequently visited states. Thus the agent is encouraged to visit those states which the discriminator is less confident to judge as visited. GAEX is easy to implement and of high training efficiency. In our experiments, we apply GAEX into DQN and the DQN-GAEX algorithm achieves convincing performance on challenging exploration problems, including the game Venture, M

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyhong2022generative

Related papers

• Gan-based Intrinsic Exploration For Sample Efficient Reinforcement Learning (2022)2.26
• Improving Policy Gradient By Exploring Under-appreciated Rewards (2016)0.00
• Off-policy Reinforcement Learning With Model-based Exploration Augmentation (2025)0.00
• Explore-go: Leveraging Exploration For Generalisation In Deep Reinforcement Learning (2024)0.00
• Neighboring State-based Exploration For Reinforcement Learning (2022)0.00
• Generative Adversarial Imagination For Sample Efficient Deep Reinforcement Learning (2019)0.00
• Never Give Up: Learning Directed Exploration Strategies (2020)0.00
• Go-explore: A New Approach For Hard-exploration Problems (2019)0.00