Why Online Reinforcement Learning Is Causal

Abstract

Reinforcement learning (RL) and causal modelling naturally complement each other. The goal of causal modelling is to predict the effects of interventions in an environment, while the goal of reinforcement learning is to select interventions that maximize the rewards the agent receives from the environment. Reinforcement learning includes the two most powerful sources of information for estimating causal relationships: temporal ordering and the ability to act on an environment. This paper examines which reinforcement learning settings we can expect to benefit from causal modelling, and how. In online learning, the agent has the ability to interact directly with their environment, and learn from exploring it. Our main argument is that in online learning, conditional probabilities are causal, and therefore offline RL is the setting where causal learning has the most potential to make a difference. Essentially, the reason is that when an agent learns from their \{\em own\} experience, ther

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyschulte2024why

Related papers

• Causal Reinforcement Learning Using Observational And Interventional Data (2021)0.00
• Learning By Doing: An Online Causal Reinforcement Learning Framework With Causal-aware Policy (2024)1.56
• Explainable Reinforcement Learning Via A Causal World Model (2023)9.03
• Explainable Reinforcement Learning Through A Causal Lens (2019)16.69
• Learning Nonlinear Causal Reductions To Explain Reinforcement Learning Policies (2025)0.00
• Unifying Causal Reinforcement Learning: Survey, Taxonomy, Algorithms And Applications (2025)0.00
• BECAUSE: Bilinear Causal Representation For Generalizable Offline Model-based Reinforcement Learning (2024)0.00
• Learning Causal Overhypotheses Through Exploration In Children And Computational Models (2022)0.00