Reset-free Reinforcement Learning With World Models

Abstract

Reinforcement learning (RL) is an appealing paradigm for training intelligent agents, enabling policy acquisition from the agent's own autonomously acquired experience. However, the training process of RL is far from automatic, requiring extensive human effort to reset the agent and environments. To tackle the challenging reset-free setting, we first demonstrate the superiority of model-based (MB) RL methods in such setting, showing that a straightforward adaptation of MBRL can outperform all the prior state-of-the-art methods while requiring less supervision. We then identify limitations inherent to this direct extension and propose a solution called model-based reset-free (MoReFree) agent, which further enhances the performance. MoReFree adapts two key mechanisms, exploration and policy learning, to handle reset-free tasks by prioritizing task-relevant states. It exhibits superior data-efficiency across various reset-free tasks without access to environmental reward or demonstrations

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyyang2024reset

Related papers

• Mind The Model, Not The Agent: The Primacy Bias In Model-based RL (2023)0.00
• Morel : Model-based Offline Reinforcement Learning (2020)0.00
• Leave No Trace: Learning To Reset For Safe And Autonomous Reinforcement Learning (2017)0.00
• Policy-driven World Model Adaptation For Robust Offline Model-based Reinforcement Learning (2025)0.00
• Model-free Reinforcement Learning For Model-based Control: Towards Safe, Interpretable And Sample-efficient Agents (2025)0.00
• PWM: Policy Learning With Multi-task World Models (2024)0.00
• Reinforcement Learning With World Model (2019)0.00
• Sample-efficient And Safe Deep Reinforcement Learning Via Reset Deep Ensemble Agents (2023)0.00