Diverse Policies Converge In Reward-free Markov Decision Processe

Abstract

Reinforcement learning has achieved great success in many decision-making tasks, and traditional reinforcement learning algorithms are mainly designed for obtaining a single optimal solution. However, recent works show the importance of developing diverse policies, which makes it an emerging research topic. Despite the variety of diversity reinforcement learning algorithms that have emerged, none of them theoretically answer the question of how the algorithm converges and how efficient the algorithm is. In this paper, we provide a unified diversity reinforcement learning framework and investigate the convergence of training diverse policies. Under such a framework, we also propose a provably efficient diversity reinforcement learning algorithm. Finally, we verify the effectiveness of our method through numerical experiments.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keylin2023diverse

Related papers

• DGPO: Discovering Multiple Strategies With Diversity-guided Policy Optimization (2022)2.26
• Effective Diversity In Population Based Reinforcement Learning (2020)0.00
• Unifying Behavioral And Response Diversity For Open-ended Learning In Zero-sum Games (2021)0.00
• Diversity-inducing Policy Gradient: Using Maximum Mean Discrepancy To Find A Set Of Diverse Policies (2019)8.35
• Phasic Diversity Optimization For Population-based Reinforcement Learning (2024)0.00
• Maximum Entropy Diverse Exploration: Disentangling Maximum Entropy Reinforcement Learning (2019)0.00
• Diverse Exploration For Fast And Safe Policy Improvement (2018)4.52
• Learning Self-imitating Diverse Policies (2018)0.00