Off-policy Reinforcement Learning With Model-based Exploration Augmentation

Abstract

Exploration is fundamental to reinforcement learning (RL), as it determines how effectively an agent discovers and exploits the underlying structure of its environment to achieve optimal performance. Existing exploration methods generally fall into two categories: active exploration and passive exploration. The former introduces stochasticity into the policy but struggles in high-dimensional environments, while the latter adaptively prioritizes transitions in the replay buffer to enhance exploration, yet remains constrained by limited sample diversity. To address the limitation in passive exploration, we propose Modelic Generative Exploration (MoGE), which augments exploration through the generation of under-explored critical states and synthesis of dynamics-consistent experiences through transition models. MoGE is composed of two components: (1) a diffusion-based generator that synthesizes critical states under the guidance of a utility function evaluating each state's potential influ

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keywang2025off

Related papers

• Learning Off-policy With Model-based Intrinsic Motivation For Active Online Exploration (2024)0.00
• PC-MLP: Model-based Reinforcement Learning With Policy Cover Guided Exploration (2021)0.00
• Improving Policy Gradient By Exploring Under-appreciated Rewards (2016)0.00
• Generative Adversarial Exploration For Reinforcement Learning (2022)0.00
• Llm-explorer: A Plug-in Reinforcement Learning Policy Exploration Enhancement Driven By Large Language Models (2025)0.00
• Model-based Active Exploration (2018)0.00
• Investigating The Role Of Model-based Learning In Exploration And Transfer (2023)0.00
• Sample Efficient Reinforcement Learning Via Model-ensemble Exploration And Exploitation (2021)0.00