Causal Confusion In Imitation Learning

Abstract

Behavioral cloning reduces policy learning to supervised learning by training a discriminative model to predict expert actions given observations. Such discriminative models are non-causal: the training procedure is unaware of the causal structure of the interaction between the expert and the environment. We point out that ignoring causality is particularly damaging because of the distributional shift in imitation learning. In particular, it leads to a counter-intuitive "causal misidentification" phenomenon: access to more information can yield worse performance. We investigate how this problem arises, and propose a solution to combat it through targeted interventions---either environment interaction or expert queries---to determine the correct causal model. We show that causal misidentification occurs in several benchmark control domains as well as realistic driving settings, and validate our solution against DAgger and other baselines and ablations.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keydehaan2019causal

Related papers

• Causal Imitation Learning With Unobserved Confounders (2022)0.00
• Self-supervised Adversarial Imitation Learning (2023)0.00
• Causal Imitation Learning Under Measurement Error And Distribution Shift (2026)0.00
• Causal Imitation Learning Under Expert-observable And Expert-unobservable Confounding (2025)0.00
• Causal Imitation Learning Under Temporally Correlated Noise (2022)0.00
• Invariant Causal Imitation Learning For Generalizable Policies (2023)0.00
• Feedback In Imitation Learning: The Three Regimes Of Covariate Shift (2021)0.00
• Sequential Causal Imitation Learning With Unobserved Confounders (2022)0.00