Learning To Reach Goals Via Iterated Supervised Learning

Abstract

Current reinforcement learning (RL) algorithms can be brittle and difficult to use, especially when learning goal-reaching behaviors from sparse rewards. Although supervised imitation learning provides a simple and stable alternative, it requires access to demonstrations from a human supervisor. In this paper, we study RL algorithms that use imitation learning to acquire goal reaching policies from scratch, without the need for expert demonstrations or a value function. In lieu of demonstrations, we leverage the property that any trajectory is a successful demonstration for reaching the final state in that same trajectory. We propose a simple algorithm in which an agent continually relabels and imitates the trajectories it generates to progressively learn goal-reaching behaviors from scratch. Each iteration, the agent collects new trajectories using the latest policy, and maximizes the likelihood of the actions along these trajectories under the goal that was actually reached, so as to

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyghosh2019learning

Related papers

• Reward-conditioned Policies (2019)0.00
• Learning Self-imitating Diverse Policies (2018)0.00
• On-policy Robot Imitation Learning From A Converging Supervisor (2019)0.00
• TGRL: An Algorithm For Teacher Guided Reinforcement Learning (2023)0.00
• Learn The Ropes, Then Trust The Wins: Self-imitation With Progressive Exploration For Agentic Reinforcement Learning (2025)0.00
• Self-supervised Goal-reaching Results In Multi-agent Cooperation And Exploration (2025)0.00
• Imitate The Good And Avoid The Bad: An Incremental Approach To Safe Reinforcement Learning (2023)0.00
• GRAIL: Goal Recognition Alignment Through Imitation Learning (2026)0.00