A Pragmatic Look At Deep Imitation Learning

Abstract

The introduction of the generative adversarial imitation learning (GAIL) algorithm has spurred the development of scalable imitation learning approaches using deep neural networks. Many of the algorithms that followed used a similar procedure, combining on-policy actor-critic algorithms with inverse reinforcement learning. More recently there have been an even larger breadth of approaches, most of which use off-policy algorithms. However, with the breadth of algorithms, everything from datasets to base reinforcement learning algorithms to evaluation settings can vary, making it difficult to fairly compare them. In this work we re-implement 6 different IL algorithms, updating 3 of them to be off-policy, base them on a common off-policy algorithm (SAC), and evaluate them on a widely-used expert trajectory dataset (D4RL) for the most common benchmark (MuJoCo). After giving all algorithms the same hyperparameter optimisation budget, we compare their results for a range of expert trajectori

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyarulkumaran2021a

Related papers

• Non-adversarial Imitation Learning And Its Connections To Adversarial Methods (2020)0.00
• C-GAIL: Stabilizing Generative Adversarial Imitation Learning With Control Theory (2024)0.00
• When Will Generative Adversarial Imitation Learning Algorithms Attain Global Convergence (2020)0.00
• Augmenting GAIL With BC For Sample Efficient Imitation Learning (2020)0.00
• Discriminator-actor-critic: Addressing Sample Inefficiency And Reward Bias In Adversarial Imitation Learning (2018)0.00
• Co-adaptation Of Algorithmic And Implementational Innovations In Inference-based Deep Reinforcement Learning (2021)0.00
• \(f\)-gail: Learning \(f\)-divergence For Generative Adversarial Imitation Learning (2020)0.00
• Lipschitzness Is All You Need To Tame Off-policy Generative Adversarial Imitation Learning (2020)7.81