Learning By Doing: An Online Causal Reinforcement Learning Framework With Causal-aware Policy

Abstract

As a key component to intuitive cognition and reasoning solutions in human intelligence, causal knowledge provides great potential for reinforcement learning (RL) agents' interpretability towards decision-making by helping reduce the searching space. However, there is still a considerable gap in discovering and incorporating causality into RL, which hinders the rapid development of causal RL. In this paper, we consider explicitly modeling the generation process of states with the causal graphical model, based on which we augment the policy. We formulate the causal structure updating into the RL interaction process with active intervention learning of the environment. To optimize the derived objective, we propose a framework with theoretical performance guarantees that alternates between two steps: using interventions for causal structure learning during exploration and using the learned causal structure for policy guidance during exploitation. Due to the lack of public benchmarks that

Stats

• citations0
• S2 citations—
• github stars5
• HF likes0
• heat score1.56
• arxiv keycai2024learning

Related papers

• Learning Nonlinear Causal Reductions To Explain Reinforcement Learning Policies (2025)0.00
• Why Online Reinforcement Learning Is Causal (2024)0.00
• Causal Reinforcement Learning Using Observational And Interventional Data (2021)0.00
• Explainable Reinforcement Learning Via A Causal World Model (2023)9.03
• A Roadmap Towards Improving Multi-agent Reinforcement Learning With Causal Discovery And Inference (2025)0.00
• Unifying Causal Reinforcement Learning: Survey, Taxonomy, Algorithms And Applications (2025)0.00
• Learning Causal Overhypotheses Through Exploration In Children And Computational Models (2022)0.00
• Explainable Reinforcement Learning Through A Causal Lens (2019)16.69