Causal Reinforcement Learning Using Observational And Interventional Data

Abstract

Learning efficiently a causal model of the environment is a key challenge of model-based RL agents operating in POMDPs. We consider here a scenario where the learning agent has the ability to collect online experiences through direct interactions with the environment (interventional data), but has also access to a large collection of offline experiences, obtained by observing another agent interacting with the environment (observational data). A key ingredient, that makes this situation non-trivial, is that we allow the observed agent to interact with the environment based on hidden information, which is not observed by the learning agent. We then ask the following questions: can the online and offline experiences be safely combined for learning a causal model ? And can we expect the offline experiences to improve the agent's performances ? To answer these questions, we import ideas from the well-established causal framework of do-calculus, and we express model-based reinforcement lear

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keygasse2021causal

Related papers

• Why Online Reinforcement Learning Is Causal (2024)0.00
• Learning By Doing: An Online Causal Reinforcement Learning Framework With Causal-aware Policy (2024)1.56
• Causal Deep Reinforcement Learning Using Observational Data (2022)5.84
• Provably Efficient Causal Reinforcement Learning With Confounded Observational Data (2020)0.00
• Learning Causal Overhypotheses Through Exploration In Children And Computational Models (2022)0.00
• Learning Nonlinear Causal Reductions To Explain Reinforcement Learning Policies (2025)0.00
• Resolving Spurious Correlations In Causal Models Of Environments Via Interventions (2020)0.00
• Explainable Reinforcement Learning Via A Causal World Model (2023)9.03