\(f\)-gail: Learning \(f\)-divergence For Generative Adversarial Imitation Learning

Abstract

Imitation learning (IL) aims to learn a policy from expert demonstrations that minimizes the discrepancy between the learner and expert behaviors. Various imitation learning algorithms have been proposed with different pre-determined divergences to quantify the discrepancy. This naturally gives rise to the following question: Given a set of expert demonstrations, which divergence can recover the expert policy more accurately with higher data efficiency? In this work, we propose \(f\)-GAIL, a new generative adversarial imitation learning (GAIL) model, that automatically learns a discrepancy measure from the \(f\)-divergence family as well as a policy capable of producing expert-like behaviors. Compared with IL baselines with various predefined divergence measures, \(f\)-GAIL learns better policies with higher data efficiency in six physics-based control tasks.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyzhang2020f

Related papers

• C-GAIL: Stabilizing Generative Adversarial Imitation Learning With Control Theory (2024)0.00
• When Will Generative Adversarial Imitation Learning Algorithms Attain Global Convergence (2020)0.00
• A Pragmatic Look At Deep Imitation Learning (2021)0.00
• Provably Efficient Generative Adversarial Imitation Learning For Online And Offline Setting With Linear Function Approximation (2021)0.00
• Diffail: Diffusion Adversarial Imitation Learning (2023)9.13
• \(f\)-policy Gradients: A General Framework For Goal Conditioned RL Using \(f\)-divergences (2023)0.00
• The \(f\)-divergence Reinforcement Learning Framework (2021)0.00
• F-divergence Constrained Policy Improvement (2017)0.00