Accelerating Imitation Learning With Predictive Models

Abstract

Sample efficiency is critical in solving real-world reinforcement learning problems, where agent-environment interactions can be costly. Imitation learning from expert advice has proved to be an effective strategy for reducing the number of interactions required to train a policy. Online imitation learning, which interleaves policy evaluation and policy optimization, is a particularly effective technique with provable performance guarantees. In this work, we seek to further accelerate the convergence rate of online imitation learning, thereby making it more sample efficient. We propose two model-based algorithms inspired by Follow-the-Leader (FTL) with prediction: MoBIL-VI based on solving variational inequalities and MoBIL-Prox based on stochastic first-order updates. These two methods leverage a model to predict future gradients to speed up policy learning. When the model oracle is learned online, these algorithms can provably accelerate the best known convergence rate up to an order

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keycheng2018accelerating

Related papers

• Explaining Fast Improvement In Online Imitation Learning (2020)0.00
• Minimax Optimal Online Imitation Learning Via Replay Estimation (2022)0.00
• Online Adaptation For Enhancing Imitation Learning Policies (2024)0.00
• Proximal Point Imitation Learning (2022)0.00
• Policy Improvement Via Imitation Of Multiple Oracles (2020)0.00
• Mimicking Better By Matching The Approximate Action Distribution (2023)0.00
• Learning Off-policy With Model-based Intrinsic Motivation For Active Online Exploration (2024)0.00
• Efficient Offline Reinforcement Learning: First Imitate, Then Improve (2024)1.91