Intrinsically Motivated Self-supervised Learning In Reinforcement Learning

Abstract

In vision-based reinforcement learning (RL) tasks, it is prevalent to assign auxiliary tasks with a surrogate self-supervised loss so as to obtain more semantic representations and improve sample efficiency. However, abundant information in self-supervised auxiliary tasks has been disregarded, since the representation learning part and the decision-making part are separated. To sufficiently utilize information in auxiliary tasks, we present a simple yet effective idea to employ self-supervised loss as an intrinsic reward, called Intrinsically Motivated Self-Supervised learning in Reinforcement learning (IM-SSR). We formally show that the self-supervised loss can be decomposed as exploration for novel states and robustness improvement from nuisance elimination. IM-SSR can be effortlessly plugged into any reinforcement learning with self-supervised auxiliary objectives with nearly no additional cost. Combined with IM-SSR, the previous underlying algorithms achieve salient improvements on

Stats

• citations2
• S2 citations—
• github stars0
• HF likes0
• heat score3.58
• arxiv keyzhao2021intrinsically

Related papers

• Loss Is Its Own Reward: Self-supervision For Reinforcement Learning (2016)0.00
• Return-based Contrastive Representation Learning For Reinforcement Learning (2021)12.17
• A Self-supervised Auxiliary Loss For Deep RL In Partially Observable Settings (2021)0.00
• Improving Reinforcement Learning Efficiency With Auxiliary Tasks In Non-visual Environments: A Comparison (2023)2.26
• When Does Self-prediction Help? Understanding Auxiliary Tasks In Reinforcement Learning (2024)0.00
• An Information-theoretic Perspective On Intrinsic Motivation In Reinforcement Learning: A Survey (2022)11.93
• Rlexplore: Accelerating Research In Intrinsically-motivated Reinforcement Learning (2024)5.33
• Self-supervised Exploration Via Temporal Inconsistency In Reinforcement Learning (2022)3.58