Causally Correct Partial Models For Reinforcement Learning

Abstract

In reinforcement learning, we can learn a model of future observations and rewards, and use it to plan the agent's next actions. However, jointly modeling future observations can be computationally expensive or even intractable if the observations are high-dimensional (e.g. images). For this reason, previous works have considered partial models, which model only part of the observation. In this paper, we show that partial models can be causally incorrect: they are confounded by the observations they don't model, and can therefore lead to incorrect planning. To address this, we introduce a general family of partial models that are provably causally correct, yet remain fast because they do not need to fully model future observations.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyrezende2020causally

Related papers

• Learning Causal State Representations Of Partially Observable Environments (2019)0.00
• Provable Representation With Efficient Planning For Partial Observable Reinforcement Learning (2023)0.00
• Provably Efficient Reinforcement Learning In Partially Observable Dynamical Systems (2022)0.00
• Learning Causal States Under Partial Observability And Perturbation (2025)0.00
• Learning Dynamics Model In Reinforcement Learning By Incorporating The Long Term Future (2019)0.00
• Partial Models For Building Adaptive Model-based Reinforcement Learning Agents (2024)0.00
• Causal Reinforcement Learning Using Observational And Interventional Data (2021)0.00
• Learning Nonlinear Causal Reductions To Explain Reinforcement Learning Policies (2025)0.00