Exploration-enhanced POLITEX

Abstract

We study algorithms for average-cost reinforcement learning problems with value function approximation. Our starting point is the recently proposed POLITEX algorithm, a version of policy iteration where the policy produced in each iteration is near-optimal in hindsight for the sum of all past value function estimates. POLITEX has sublinear regret guarantees in uniformly-mixing MDPs when the value estimation error can be controlled, which can be satisfied if all policies sufficiently explore the environment. Unfortunately, this assumption is often unrealistic. Motivated by the rapid growth of interest in developing policies that learn to explore their environment in the lack of rewards (also known as no-reward learning), we replace the previous assumption that all policies explore the environment with that a single, sufficiently exploring policy is available beforehand. The main contribution of the paper is the modification of POLITEX to incorporate such an exploration policy in a way t

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyabbasiyadkori2019exploration

Related papers

• Improved Regret Bound And Experience Replay In Regularized Policy Iteration (2021)0.00
• Provably Efficient Exploration In Policy Optimization (2019)0.00
• Conservative Exploration In Reinforcement Learning (2020)0.00
• Cautiously Optimistic Policy Optimization And Exploration With Linear Function Approximation (2021)0.00
• Near-optimal Conservative Exploration In Reinforcement Learning Under Episode-wise Constraints (2023)0.00
• Exploration Conscious Reinforcement Learning Revisited (2018)0.00
• Improved Bounds For Reward-agnostic And Reward-free Exploration (2026)0.00
• Proximal Policy Optimization With Adaptive Exploration (2024)0.00