Self-supervised Reinforcement Learning That Transfers Using Random Features

Abstract

Model-free reinforcement learning algorithms have exhibited great potential in solving single-task sequential decision-making problems with high-dimensional observations and long horizons, but are known to be hard to generalize across tasks. Model-based RL, on the other hand, learns task-agnostic models of the world that naturally enables transfer across different reward functions, but struggles to scale to complex environments due to the compounding error. To get the best of both worlds, we propose a self-supervised reinforcement learning method that enables the transfer of behaviors across tasks with different rewards, while circumventing the challenges of model-based RL. In particular, we show self-supervised pre-training of model-free reinforcement learning with a number of random features as rewards allows implicit modeling of long-horizon environment dynamics. Then, planning techniques like model-predictive control using these implicit models enable fast adaptation to problems wi

Stats

• citations1
• S2 citations—
• github stars0
• HF likes0
• heat score2.26
• arxiv keychen2023self

Related papers

• On The Feasibility Of Cross-task Transfer With Model-based Reinforcement Learning (2022)0.00
• Uncertainty-aware Transfer Across Tasks Using Hybrid Model-based Successor Feature Reinforcement Learning (2023)5.24
• Transfer With Model Features In Reinforcement Learning (2018)0.00
• Self-adapting Goals Allow Transfer Of Predictive Models To New Tasks (2019)2.26
• Generating Automatic Curricula Via Self-supervised Active Domain Randomization (2020)0.00
• Successor Features For Transfer In Reinforcement Learning (2016)0.00
• Advantages And Limitations Of Using Successor Features For Transfer In Reinforcement Learning (2017)0.00
• Contextual Pre-planning On Reward Machine Abstractions For Enhanced Transfer In Deep Reinforcement Learning (2023)2.26