Beyond-expert Performance With Limited Demonstrations: Efficient Imitation Learning With Double Exploration

Abstract

Imitation learning is a central problem in reinforcement learning where the goal is to learn a policy that mimics the expert's behavior. In practice, it is often challenging to learn the expert policy from a limited number of demonstrations accurately due to the complexity of the state space. Moreover, it is essential to explore the environment and collect data to achieve beyond-expert performance. To overcome these challenges, we propose a novel imitation learning algorithm called Imitation Learning with Double Exploration (ILDE), which implements exploration in two aspects: (1) optimistic policy optimization via an exploration bonus that rewards state-action pairs with high uncertainty to potentially improve the convergence to the expert policy, and (2) curiosity-driven exploration of the states that deviate from the demonstration trajectories to potentially yield beyond-expert performance. Empirically, we demonstrate that ILDE outperforms the state-of-the-art imitation learning algo

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyzhao2025beyond

Related papers

• Good Better Best: Self-motivated Imitation Learning For Noisy Demonstrations (2023)0.00
• Co-imitation Learning Without Expert Demonstration (2021)0.00
• Toward The Fundamental Limits Of Imitation Learning (2020)0.00
• A Dual Approach To Imitation Learning From Observations With Offline Datasets (2024)0.00
• A Bayesian Solution To The Imitation Gap (2024)0.00
• State-only Imitation With Transition Dynamics Mismatch (2020)0.00
• Matching Multiple Experts: On The Exploitability Of Multi-agent Imitation Learning (2026)0.00
• Plan Your Target And Learn Your Skills: Transferable State-only Imitation Learning Via Decoupled Policy Optimization (2022)0.00