Nuclear Norm Maximization Based Curiosity-driven Learning

Abstract

To handle the sparsity of the extrinsic rewards in reinforcement learning, researchers have proposed intrinsic reward which enables the agent to learn the skills that might come in handy for pursuing the rewards in the future, such as encouraging the agent to visit novel states. However, the intrinsic reward can be noisy due to the undesirable environment's stochasticity and directly applying the noisy value predictions to supervise the policy is detrimental to improve the learning performance and efficiency. Moreover, many previous studies employ \(\ell^2\) norm or variance to measure the exploration novelty, which will amplify the noise due to the square operation. In this paper, we address aforementioned challenges by proposing a novel curiosity leveraging the nuclear norm maximization (NNM), which can quantify the novelty of exploring the environment more accurately while providing high-tolerance to the noise and outliers. We conduct extensive experiments across a variety of benchm

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keychen2022nuclear

Related papers

• Self-supervised Exploration Via Temporal Inconsistency In Reinforcement Learning (2022)3.58
• Curiosity-driven Multi-agent Exploration With Mixed Objectives (2022)0.00
• Information Content Exploration (2023)0.00
• Intrinsic Rewards For Exploration Without Harm From Observational Noise: A Simulation Study Based On The Free Energy Principle (2024)0.00
• Intrinsic Reward Policy Optimization For Sparse-reward Environments (2026)0.00
• Curiosity-driven Exploration In Sparse-reward Multi-agent Reinforcement Learning (2023)0.00
• Beyond Surprise: Improving Exploration Through Surprise Novelty (2023)0.00
• Redeeming Intrinsic Rewards Via Constrained Optimization (2022)0.00