Interactive And Hybrid Imitation Learning: Provably Beating Behavior Cloning

Abstract

Imitation learning (IL) is a paradigm for learning sequential decision making policies from experts, leveraging offline demonstrations, interactive annotations, or both. Recent advances show that when annotation cost is tallied per trajectory, Behavior Cloning (BC) which relies solely on offline demonstrations cannot be improved in general, leaving limited conditions for interactive methods such as DAgger to help. We revisit this conclusion and prove that when the annotation cost is measured per state, algorithms using interactive annotations can provably outperform BC. Specifically: (1) we show that Stagger, a one sample per round variant of DAgger, provably beats BC under low recovery cost settings; (2) we initiate the study of hybrid IL where the agent learns from offline demonstrations and interactive annotations. We propose Warm Stagger whose learning guarantee is not much worse than using either data source alone. Furthermore, motivated by compounding error and cold start problem

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyli2024interactive

Related papers

• Is Behavior Cloning All You Need? Understanding Horizon In Imitation Learning (2024)0.00
• Swarm Behavior Cloning (2024)0.00
• RLIF: Interactive Imitation Learning As Reinforcement Learning (2023)0.00
• Self-supervised Adversarial Imitation Learning (2023)0.00
• State-only Imitation With Transition Dynamics Mismatch (2020)0.00
• Augmenting GAIL With BC For Sample Efficient Imitation Learning (2020)0.00
• When Should We Prefer Offline Reinforcement Learning Over Behavioral Cloning? (2022)0.00
• Causal Imitation Learning Under Measurement Error And Distribution Shift (2026)0.00