The Pitfalls Of Imitation Learning When Actions Are Continuous

Abstract

We study the problem of imitating an expert demonstrator in a discrete-time, continuous state-and-action control system. We show that, even if the dynamics satisfy a control-theoretic property called exponential stability (i.e. the effects of perturbations decay exponentially quickly), and the expert is smooth and deterministic, any smooth, deterministic imitator policy necessarily suffers error on execution that is exponentially larger, as a function of problem horizon, than the error under the distribution of expert training data. Our negative result applies to any algorithm which learns solely from expert data, including both behavior cloning and offline-RL algorithms, unless the algorithm produces highly "improper" imitator policies--those which are non-smooth, non-Markovian, or which exhibit highly state-dependent stochasticity--or unless the expert trajectory distribution is sufficiently "spread." We provide experimental evidence of the benefits of these more complex policy param

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keysimchowitz2025the

Related papers

• Toward The Fundamental Limits Of Imitation Learning (2020)0.00
• Sequence Model Imitation Learning With Unobserved Contexts (2022)0.00
• State-only Imitation With Transition Dynamics Mismatch (2020)0.00
• Causal Confusion In Imitation Learning (2019)0.00
• Minimax Optimal Online Imitation Learning Via Replay Estimation (2022)0.00
• Causal Imitation Learning Under Temporally Correlated Noise (2022)0.00
• Offline Imitation Learning By Controlling The Effective Planning Horizon (2024)0.00
• Fully General Online Imitation Learning (2021)0.00