Sample-efficient And Safe Deep Reinforcement Learning Via Reset Deep Ensemble Agents

Abstract

Deep reinforcement learning (RL) has achieved remarkable success in solving complex tasks through its integration with deep neural networks (DNNs) as function approximators. However, the reliance on DNNs has introduced a new challenge called primacy bias, whereby these function approximators tend to prioritize early experiences, leading to overfitting. To mitigate this primacy bias, a reset method has been proposed, which performs periodic resets of a portion or the entirety of a deep RL agent while preserving the replay buffer. However, the use of the reset method can result in performance collapses after executing the reset, which can be detrimental from the perspective of safe RL and regret minimization. In this paper, we propose a new reset-based method that leverages deep ensemble learning to address the limitations of the vanilla reset method and enhance sample efficiency. The proposed method is evaluated through various experiments including those in the domain of safe RL. Numer

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keykim2023sample

Related papers

• Leave No Trace: Learning To Reset For Safe And Autonomous Reinforcement Learning (2017)0.00
• Reset-free Reinforcement Learning With World Models (2024)0.00
• Episodic Reinforcement Learning With Expanded State-reward Space (2024)0.00
• Stabilising Experience Replay For Deep Multi-agent Reinforcement Learning (2017)0.00
• SUNRISE: A Simple Unified Framework For Ensemble Learning In Deep Reinforcement Learning (2020)0.00
• Deep Reinforcement Learning At The Edge Of The Statistical Precipice (2021)0.00
• Human-inspired Framework To Accelerate Reinforcement Learning (2023)0.00
• Deep Reinforcement Learning In A Handful Of Trials Using Probabilistic Dynamics Models (2018)0.00