Task Phasing: Automated Curriculum Learning From Demonstrations

Abstract

Applying reinforcement learning (RL) to sparse reward domains is notoriously challenging due to insufficient guiding signals. Common RL techniques for addressing such domains include (1) learning from demonstrations and (2) curriculum learning. While these two approaches have been studied in detail, they have rarely been considered together. This paper aims to do so by introducing a principled task phasing approach that uses demonstrations to automatically generate a curriculum sequence. Using inverse RL from (suboptimal) demonstrations we define a simple initial task. Our task phasing approach then provides a framework to gradually increase the complexity of the task all the way to the target task, while retuning the RL agent in each phasing iteration. Two approaches for phasing are considered: (1) gradually increasing the proportion of time steps an RL agent is in control, and (2) phasing out a guiding informative reward function. We present conditions that guarantee the convergence

Stats

• citations4
• S2 citations—
• github stars0
• HF likes0
• heat score5.24
• arxiv keybajaj2022task

Related papers

• Reverse Forward Curriculum Learning For Extreme Sample And Demonstration Efficiency In Reinforcement Learning (2024)0.00
• Causal-paced Deep Reinforcement Learning (2025)0.00
• Proximal Curriculum With Task Correlations For Deep Reinforcement Learning (2024)0.00
• Curriculum Learning For Reinforcement Learning Domains: A Framework And Survey (2020)0.00
• Generating Automatic Curricula Via Self-supervised Active Domain Randomization (2020)0.00
• Learning Curriculum Policies For Reinforcement Learning (2018)5.24
• Backplay: "man Muss Immer Umkehren" (2018)0.00
• Curriculum Learning With A Progression Function (2020)0.00