Forward-backward Reinforcement Learning

Abstract

Goals for reinforcement learning problems are typically defined through hand-specified rewards. To design such problems, developers of learning algorithms must inherently be aware of what the task goals are, yet we often require agents to discover them on their own without any supervision beyond these sparse rewards. While much of the power of reinforcement learning derives from the concept that agents can learn with little guidance, this requirement greatly burdens the training process. If we relax this one restriction and endow the agent with knowledge of the reward function, and in particular of the goal, we can leverage backwards induction to accelerate training. To achieve this, we propose training a model to learn to take imagined reversal steps from known goal states. Rather than training an agent exclusively to determine how to reach a goal while moving forwards in time, our approach travels backwards to jointly predict how we got there. We evaluate our work in Gridworld and To

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyedwards2018forward

Related papers

• Backward Curriculum Reinforcement Learning (2022)0.00
• Backward Learning For Goal-conditioned Policies (2023)0.00
• Learning What To Do By Simulating The Past (2021)0.00
• Backplay: "man Muss Immer Umkehren" (2018)0.00
• There Is No Turning Back: A Self-supervised Approach For Reversibility-aware Reinforcement Learning (2021)0.00
• Learning To Predict Without Looking Ahead: World Models Without Forward Prediction (2019)0.00
• Automatic Goal Generation For Reinforcement Learning Agents (2017)0.00
• Reverse Forward Curriculum Learning For Extreme Sample And Demonstration Efficiency In Reinforcement Learning (2024)0.00