Extra: Transfer-guided Exploration

Abstract

In this work we present a novel approach for transfer-guided exploration in reinforcement learning that is inspired by the human tendency to leverage experiences from similar encounters in the past while navigating a new task. Given an optimal policy in a related task-environment, we show that its bisimulation distance from the current task-environment gives a lower bound on the optimal advantage of state-action pairs in the current task-environment. Transfer-guided Exploration (ExTra) samples actions from a Softmax distribution over these lower bounds. In this way, actions with potentially higher optimum advantage are sampled more frequently. In our experiments on gridworld environments, we demonstrate that given access to an optimal policy in a related task-environment, ExTra can outperform popular domain-specific exploration strategies viz. epsilon greedy, Model-Based Interval Estimation - Exploration Bonus (MBIE-EB), Pursuit and Boltzmann in rate of convergence. We further show tha

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keysantara2019extra

Related papers

• Investigating The Role Of Model-based Learning In Exploration And Transfer (2023)0.00
• The Role Of Exploration For Task Transfer In Reinforcement Learning (2022)0.00
• Exploration Conscious Reinforcement Learning Revisited (2018)0.00
• Exploration Via Elliptical Episodic Bonuses (2022)3.58
• Exploration In Knowledge Transfer Utilizing Reinforcement Learning (2024)0.00
• Go-explore: A New Approach For Hard-exploration Problems (2019)0.00
• Is Exploration All You Need? Effective Exploration Characteristics For Transfer In Reinforcement Learning (2024)0.00
• Never Give Up: Learning Directed Exploration Strategies (2020)0.00