Wasserstein Distance Maximizing Intrinsic Control

Abstract

This paper deals with the problem of learning a skill-conditioned policy that acts meaningfully in the absence of a reward signal. Mutual information based objectives have shown some success in learning skills that reach a diverse set of states in this setting. These objectives include a KL-divergence term, which is maximized by visiting distinct states even if those states are not far apart in the MDP. This paper presents an approach that rewards the agent for learning skills that maximize the Wasserstein distance of their state visitation from the start state of the skill. It shows that such an objective leads to a policy that covers more distance in the MDP than diversity based objectives, and validates the results on a variety of Atari environments.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keydurugkar2021wasserstein

Related papers

• Relative Variational Intrinsic Control (2020)0.00
• Learning To Optimize Via Wasserstein Deep Inverse Optimal Control (2018)0.00
• Diversity Is All You Need: Learning Skills Without A Reward Function (2018)0.00
• Learning To Score Behaviors For Guided Policy Optimization (2019)0.00
• Focused Skill Discovery: Learning To Control Specific State Variables While Minimizing Side Effects (2025)0.00
• Reinforcement Learning With Wasserstein Distance Regularisation, With Applications To Multipolicy Learning (2018)0.00
• Primal Wasserstein Imitation Learning (2020)0.00
• Generative Intrinsic Optimization: Intrinsic Control With Model Learning (2023)0.00