Exploring Unknown States With Action Balance

Abstract

Exploration is a key problem in reinforcement learning. Recently bonus-based methods have achieved considerable successes in environments where exploration is difficult such as Montezuma's Revenge, which assign additional bonuses (e.g., intrinsic rewards) to guide the agent to rarely visited states. Since the bonus is calculated according to the novelty of the next state after performing an action, we call such methods as the next-state bonus methods. However, the next-state bonus methods force the agent to pay overmuch attention in exploring known states and ignore finding unknown states since the exploration is driven by the next state already visited, which may slow the pace of finding reward in some environments. In this paper, we focus on improving the effectiveness of finding unknown states and propose action balance exploration, which balances the frequency of selecting each action at a given state and can be treated as an extension of upper confidence bound (UCB) to deep reinfo

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keysong2020exploring

Related papers

• Neighboring State-based Exploration For Reinforcement Learning (2022)0.00
• Exploration Via Elliptical Episodic Bonuses (2022)3.58
• Fast Active Learning For Pure Exploration In Reinforcement Learning (2020)0.00
• Anti-concentrated Confidence Bonuses For Scalable Exploration (2021)0.00
• Exploration And Incentives In Reinforcement Learning (2021)8.09
• Exploration In Feature Space For Reinforcement Learning (2017)0.00
• Go-explore: A New Approach For Hard-exploration Problems (2019)0.00
• Accelerating Reinforcement Learning With Value-conditional State Entropy Exploration (2023)0.00