Learning Relative Return Policies With Upside-down Reinforcement Learning

Abstract

Lately, there has been a resurgence of interest in using supervised learning to solve reinforcement learning problems. Recent work in this area has largely focused on learning command-conditioned policies. We investigate the potential of one such method -- upside-down reinforcement learning -- to work with commands that specify a desired relationship between some scalar value and the observed return. We show that upside-down reinforcement learning can learn to carry out such commands online in a tabular bandit setting and in CartPole with non-linear function approximation. By doing so, we demonstrate the power of this family of methods and open the way for their practical use under more complicated command structures.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyashley2022learning

Related papers

• Upside-down Reinforcement Learning For More Interpretable Optimal Control (2024)0.00
• All You Need Is Supervised Learning: From Imitation Learning To Meta-rl With Upside Down RL (2022)0.00
• Reward-conditioned Policies (2019)0.00
• Upside-down Reinforcement Learning Can Diverge In Stochastic Environments With Episodic Resets (2022)0.00
• Learning Interpretable Policies In Hindsight-observable Pomdps Through Partially Supervised Reinforcement Learning (2024)2.26
• Offline Hierarchical Reinforcement Learning Via Inverse Optimization (2024)0.00
• When Does Return-conditioned Supervised Learning Work For Offline Reinforcement Learning? (2022)0.00
• Replacing Rewards With Examples: Example-based Policy Search Via Recursive Classification (2021)0.00