Causal-paced Deep Reinforcement Learning

Abstract

Designing effective task sequences is crucial for curriculum reinforcement learning (CRL), where agents must gradually acquire skills by training on intermediate tasks. A key challenge in CRL is to identify tasks that promote exploration, yet are similar enough to support effective transfer. While recent approach suggests comparing tasks via their Structural Causal Models (SCMs), the method requires access to ground-truth causal structures, an unrealistic assumption in most RL settings. In this work, we propose Causal-Paced Deep Reinforcement Learning (CP-DRL), a curriculum learning framework aware of SCM differences between tasks based on interaction data approximation. This signal captures task novelty, which we combine with the agent's learnability, measured by reward gain, to form a unified objective. Empirically, CP-DRL outperforms existing curriculum methods on the Point Mass benchmark, achieving faster convergence and higher returns. CP-DRL demonstrates reduced variance with com

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keycho2025causal

Related papers

• Proximal Curriculum With Task Correlations For Deep Reinforcement Learning (2024)0.00
• Learning Curriculum Policies For Reinforcement Learning (2018)5.24
• Task Phasing: Automated Curriculum Learning From Demonstrations (2022)5.24
• Demonstration-guided Continual Reinforcement Learning In Dynamic Environments (2025)0.00
• Dynamics-adaptive Continual Reinforcement Learning Via Progressive Contextualization (2022)7.16
• CIER: A Novel Experience Replay Approach With Causal Inference In Deep Reinforcement Learning (2024)0.00
• Curriculum Learning For Reinforcement Learning Domains: A Framework And Survey (2020)0.00
• Continual Policy Distillation From Distributed Reinforcement Learning Teachers (2026)0.00