Distop: Discovering A Topological Representation To Learn Diverse And Rewarding Skills

Abstract

The optimal way for a deep reinforcement learning (DRL) agent to explore is to learn a set of skills that achieves a uniform distribution of states. Following this,we introduce DisTop, a new model that simultaneously learns diverse skills and focuses on improving rewarding skills. DisTop progressively builds a discrete topology of the environment using an unsupervised contrastive loss, a growing network and a goal-conditioned policy. Using this topology, a state-independent hierarchical policy can select where the agent has to keep discovering skills in the state space. In turn, the newly visited states allows an improved learnt representation and the learning loop continues. Our experiments emphasize that DisTop is agnostic to the ground state representation and that the agent can discover the topology of its environment whether the states are high-dimensional binary data, images, or proprioceptive inputs. We demonstrate that this paradigm is competitiveon MuJoCo benchmarks with state

Stats

• citations2
• S2 citations—
• github stars0
• HF likes0
• heat score3.58
• arxiv keyaubret2021distop

Related papers

• Disentangled Unsupervised Skill Discovery For Efficient Hierarchical Reinforcement Learning (2024)0.00
• Hypothesis-driven Skill Discovery For Hierarchical Deep Reinforcement Learning (2019)2.26
• Diversity Is All You Need: Learning Skills Without A Reward Function (2018)0.00
• Neuroevolution Is A Competitive Alternative To Reinforcement Learning For Skill Discovery (2022)0.00
• Learning More Skills Through Optimistic Exploration (2021)0.00
• Multi-agent Deep Covering Skill Discovery (2022)0.00
• MULEX: Disentangling Exploitation From Exploration In Deep RL (2019)0.00
• Diversity Through Exclusion (DTE): Niche Identification For Reinforcement Learning Through Value-decomposition (2023)0.00