Self-improving Skill Learning For Robust Skill-based Meta-reinforcement Learning

Abstract

Meta-reinforcement learning (Meta-RL) facilitates rapid adaptation to unseen tasks but faces challenges in long-horizon environments. Skill-based approaches tackle this by decomposing state-action sequences into reusable skills and employing hierarchical decision-making. However, these methods are highly susceptible to noisy offline demonstrations, leading to unstable skill learning and degraded performance. To address this, we propose Self-Improving Skill Learning (SISL), which performs self-guided skill refinement using decoupled high-level and skill improvement policies, while applying skill prioritization via maximum return relabeling to focus updates on task-relevant trajectories, resulting in robust and stable adaptation even under noisy and suboptimal data. By mitigating the effect of noise, SISL achieves reliable skill learning and consistently outperforms other skill-based meta-RL methods on diverse long-horizon tasks. Our code is available at https://github.com/epsilog/SISL.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keylee2025self

Code

• epsilog/SISL—

Related papers

• Skill-aware Mutual Information Optimisation For Generalisation In Reinforcement Learning (2024)0.00
• Skill-critic: Refining Learned Skills For Hierarchical Reinforcement Learning (2023)7.50
• Efficient Meta Reinforcement Learning For Preference-based Fast Adaptation (2022)0.00
• Skills: Adaptive Skill Sequencing For Efficient Temporally-extended Exploration (2022)0.00
• RL\(^3\): Boosting Meta Reinforcement Learning Via RL Inside RL\(^2\) (2023)0.00
• Learn The Ropes, Then Trust The Wins: Self-imitation With Progressive Exploration For Agentic Reinforcement Learning (2025)0.00
• Enhancing Online Reinforcement Learning With Meta-learned Objective From Offline Data (2025)0.00
• Can A MISL Fly? Analysis And Ingredients For Mutual Information Skill Learning (2024)0.00