All You Need Is Supervised Learning: From Imitation Learning To Meta-rl With Upside Down RL

Abstract

Upside down reinforcement learning (UDRL) flips the conventional use of the return in the objective function in RL upside down, by taking returns as input and predicting actions. UDRL is based purely on supervised learning, and bypasses some prominent issues in RL: bootstrapping, off-policy corrections, and discount factors. While previous work with UDRL demonstrated it in a traditional online RL setting, here we show that this single algorithm can also work in the imitation learning and offline RL settings, be extended to the goal-conditioned RL setting, and even the meta-RL setting. With a general agent architecture, a single UDRL agent can learn across all paradigms.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyarulkumaran2022all

Related papers

• Upside-down Reinforcement Learning For More Interpretable Optimal Control (2024)0.00
• Upside-down Reinforcement Learning Can Diverge In Stochastic Environments With Episodic Resets (2022)0.00
• Learning Relative Return Policies With Upside-down Reinforcement Learning (2022)0.00
• Dual RL: Unification And New Methods For Reinforcement And Imitation Learning (2023)0.00
• Meta-gradient Reinforcement Learning With An Objective Discovered Online (2020)0.00
• Learning To Reinforcement Learn (2016)0.00
• RL\(^3\): Boosting Meta Reinforcement Learning Via RL Inside RL\(^2\) (2023)0.00
• Discovering General Reinforcement Learning Algorithms With Adversarial Environment Design (2023)0.00