Deterministic Sequencing Of Exploration And Exploitation For Reinforcement Learning

Abstract

We propose Deterministic Sequencing of Exploration and Exploitation (DSEE) algorithm with interleaving exploration and exploitation epochs for model-based RL problems that aim to simultaneously learn the system model, i.e., a Markov decision process (MDP), and the associated optimal policy. During exploration, DSEE explores the environment and updates the estimates for expected reward and transition probabilities. During exploitation, the latest estimates of the expected reward and transition probabilities are used to obtain a robust policy with high probability. We design the lengths of the exploration and exploitation epochs such that the cumulative regret grows as a sub-linear function of time.

Stats

• citations5
• S2 citations—
• github stars0
• HF likes0
• heat score5.84
• arxiv keygupta2022deterministic

Related papers

• Maximum Entropy Diverse Exploration: Disentangling Maximum Entropy Reinforcement Learning (2019)0.00
• DOPE: Doubly Optimistic And Pessimistic Exploration For Safe Reinforcement Learning (2021)0.00
• Neighboring State-based Exploration For Reinforcement Learning (2022)0.00
• Decoupled Exploration And Exploitation Policies For Sample-efficient Reinforcement Learning (2021)0.00
• STEERING: Stein Information Directed Exploration For Model-based Reinforcement Learning (2023)0.00
• DQN With Model-based Exploration: Efficient Learning On Environments With Sparse Rewards (2019)0.00
• Sample Efficient Reinforcement Learning Via Model-ensemble Exploration And Exploitation (2021)0.00
• Diverse Exploration For Fast And Safe Policy Improvement (2018)4.52