Skills: Adaptive Skill Sequencing For Efficient Temporally-extended Exploration

Abstract

The ability to effectively reuse prior knowledge is a key requirement when building general and flexible Reinforcement Learning (RL) agents. Skill reuse is one of the most common approaches, but current methods have considerable limitations.For example, fine-tuning an existing policy frequently fails, as the policy can degrade rapidly early in training. In a similar vein, distillation of expert behavior can lead to poor results when given sub-optimal experts. We compare several common approaches for skill transfer on multiple domains including changes in task and system dynamics. We identify how existing methods can fail and introduce an alternative approach to mitigate these problems. Our approach learns to sequence existing temporally-extended skills for exploration but learns the final policy directly from the raw experience. This conceptual split enables rapid adaptation and thus efficient data collection but without constraining the final solution.It significantly outperforms many

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyvezzani2022skills

Related papers

• Neuroevolution Is A Competitive Alternative To Reinforcement Learning For Skill Discovery (2022)0.00
• When Do Skills Help Reinforcement Learning? A Theoretical Analysis Of Temporal Abstractions (2024)0.00
• Hierarchical And Interpretable Skill Acquisition In Multi-task Reinforcement Learning (2017)0.00
• Skill-critic: Refining Learned Skills For Hierarchical Reinforcement Learning (2023)7.50
• Leveraging Approximate Symbolic Models For Reinforcement Learning Via Skill Diversity (2022)0.00
• Learning More Skills Through Optimistic Exploration (2021)0.00
• Focused Skill Discovery: Learning To Control Specific State Variables While Minimizing Side Effects (2025)0.00
• AMPED: Adaptive Multi-objective Projection For Balancing Exploration And Skill Diversification (2025)0.00