Guided Exploration In Reinforcement Learning Via Monte Carlo Critic Optimization

Abstract

The class of deep deterministic off-policy algorithms is effectively applied to solve challenging continuous control problems. Current approaches commonly utilize random noise as an exploration method, which has several drawbacks, including the need for manual adjustment for a given task and the absence of exploratory calibration during the training process. We address these challenges by proposing a novel guided exploration method that uses an ensemble of Monte Carlo Critics for calculating exploratory action correction. The proposed method enhances the traditional exploration scheme by dynamically adjusting exploration. Subsequently, we present a novel algorithm that leverages the proposed exploratory module for both policy and critic modification. The presented algorithm demonstrates superior performance compared to modern reinforcement learning algorithms across a variety of problems in the DMControl suite.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keykuznetsov2022guided

Related papers

• Monte Carlo Beam Search For Actor-critic Reinforcement Learning In Continuous Control (2025)0.00
• Deep Intrinsically Motivated Exploration In Continuous Control (2022)0.00
• Efficient Exploration In Deep Reinforcement Learning: A Novel Bayesian Actor-critic Algorithm (2024)0.00
• Centralized Cooperative Exploration Policy For Continuous Control Tasks (2023)0.00
• Exploring More When It Needs In Deep Reinforcement Learning (2021)0.00
• Behavior-guided Actor-critic: Improving Exploration Via Learning Policy Behavior Representation For Deep Reinforcement Learning (2021)0.00
• Deep Exploration With Pac-bayes (2024)0.00
• Improved Exploration Through Latent Trajectory Optimization In Deep Deterministic Policy Gradient (2019)0.00