Leave No Trace: Learning To Reset For Safe And Autonomous Reinforcement Learning

Abstract

Deep reinforcement learning algorithms can learn complex behavioral skills, but real-world application of these methods requires a large amount of experience to be collected by the agent. In practical settings, such as robotics, this involves repeatedly attempting a task, resetting the environment between each attempt. However, not all tasks are easily or automatically reversible. In practice, this learning process requires extensive human intervention. In this work, we propose an autonomous method for safe and efficient reinforcement learning that simultaneously learns a forward and reset policy, with the reset policy resetting the environment for a subsequent attempt. By learning a value function for the reset policy, we can automatically determine when the forward policy is about to enter a non-reversible state, providing for uncertainty-aware safety aborts. Our experiments illustrate that proper use of the reset policy can greatly reduce the number of manual resets required to lear

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyeysenbach2017leave

Related papers

• Continual Learning Of Control Primitives: Skill Discovery Via Reset-games (2020)0.00
• Sample-efficient And Safe Deep Reinforcement Learning Via Reset Deep Ensemble Agents (2023)0.00
• Stochastic Resetting Accelerates Policy Convergence In Reinforcement Learning (2026)0.00
• Learning Without Time-based Embodiment Resets In Soft-actor Critic (2025)0.00
• Reset-free Reinforcement Learning With World Models (2024)0.00
• Safety Correction From Baseline: Towards The Risk-aware Policy In Robotics Via Dual-agent Reinforcement Learning (2022)3.58
• There Is No Turning Back: A Self-supervised Approach For Reversibility-aware Reinforcement Learning (2021)0.00
• Barc: Backward Reachability Curriculum For Robotic Reinforcement Learning (2018)10.74