Self-supervised Exploration Via Temporal Inconsistency In Reinforcement Learning

Abstract

Under sparse extrinsic reward settings, reinforcement learning has remained challenging, despite surging interests in this field. Previous attempts suggest that intrinsic reward can alleviate the issue caused by sparsity. In this article, we present a novel intrinsic reward that is inspired by human learning, as humans evaluate curiosity by comparing current observations with historical knowledge. Our method involves training a self-supervised prediction model, saving snapshots of the model parameters, and using nuclear norm to evaluate the temporal inconsistency between the predictions of different snapshots as intrinsic rewards. We also propose a variational weighting mechanism to assign weight to different snapshots in an adaptive manner. Our experimental results on various benchmark environments demonstrate the efficacy of our method, which outperforms other intrinsic reward-based methods without additional training costs and with higher noise tolerance. This work has been submitte

Stats

• citations2
• S2 citations—
• github stars0
• HF likes0
• heat score3.58
• arxiv keygao2022self

Related papers

• Nuclear Norm Maximization Based Curiosity-driven Learning (2022)0.00
• Long-term Visitation Value For Deep Exploration In Sparse Reward Reinforcement Learning (2020)7.24
• Information Content Exploration (2023)0.00
• Curiosity-driven Multi-agent Exploration With Mixed Objectives (2022)0.00
• Temporal Representations For Exploration: Learning Complex Exploratory Behavior Without Extrinsic Rewards (2026)0.00
• Intrinsic Reward Policy Optimization For Sparse-reward Environments (2026)0.00
• Rewarding Episodic Visitation Discrepancy For Exploration In Reinforcement Learning (2022)0.00
• Scheduled Intrinsic Drive: A Hierarchical Take On Intrinsically Motivated Exploration (2019)0.00