SEERL: Sample Efficient Ensemble Reinforcement Learning

Abstract

Ensemble learning is a very prevalent method employed in machine learning. The relative success of ensemble methods is attributed to their ability to tackle a wide range of instances and complex problems that require different low-level approaches. However, ensemble methods are relatively less popular in reinforcement learning owing to the high sample complexity and computational expense involved in obtaining a diverse ensemble. We present a novel training and model selection framework for model-free reinforcement algorithms that use ensembles of policies obtained from a single training run. These policies are diverse in nature and are learned through directed perturbation of the model parameters at regular intervals. We show that learning and selecting an adequately diverse set of policies is required for a good ensemble while extreme diversity can prove detrimental to overall performance. Selection of an adequately diverse set of policies is done through our novel policy selection fr

Stats

• citations1
• S2 citations—
• github stars0
• HF likes0
• heat score2.26
• arxiv keysaphal2020seerl

Related papers

• Towards Applicable Reinforcement Learning: Improving The Generalization And Sample Efficiency With Policy Ensemble (2022)9.23
• Sample Efficient Reinforcement Learning Via Model-ensemble Exploration And Exploitation (2021)0.00
• MEPG: A Minimalist Ensemble Policy Gradient Framework For Deep Reinforcement Learning (2021)0.00
• Sample Complexity Of Reinforcement Learning Using Linearly Combined Model Ensembles (2019)0.00
• Sample-efficient Reinforcement Learning With Stochastic Ensemble Value Expansion (2018)0.00
• Efficient Reinforcement Learning From Demonstration Using Local Ensemble And Reparameterization With Split And Merge Of Expert Policies (2022)0.00
• Epopt: Learning Robust Neural Network Policies Using Model Ensembles (2016)0.00
• Collaborative Evolutionary Reinforcement Learning (2019)0.00