Discriminator-actor-critic: Addressing Sample Inefficiency And Reward Bias In Adversarial Imitation Learning

Abstract

We identify two issues with the family of algorithms based on the Adversarial Imitation Learning framework. The first problem is implicit bias present in the reward functions used in these algorithms. While these biases might work well for some environments, they can also lead to sub-optimal behavior in others. Secondly, even though these algorithms can learn from few expert demonstrations, they require a prohibitively large number of interactions with the environment in order to imitate the expert for many real-world applications. In order to address these issues, we propose a new algorithm called Discriminator-Actor-Critic that uses off-policy Reinforcement Learning to reduce policy-environment interaction sample complexity by an average factor of 10. Furthermore, since our reward function is designed to be unbiased, we can apply our algorithm to many problems without making any task-specific adjustments.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keykostrikov2018discriminator

Related papers

• On Discovering Algorithms For Adversarial Imitation Learning (2025)0.00
• Deterministic And Discriminative Imitation (d2-imitation): Revisiting Adversarial Imitation For Sample Efficiency (2021)0.00
• Sample-efficient Model-free Reinforcement Learning With Off-policy Critics (2019)9.60
• Imitating Opponent To Win: Adversarial Policy Imitation Learning In Two-player Competitive Games (2022)0.00
• Rethinking Adversarial Inverse Reinforcement Learning: Policy Imitation, Transferable Reward Recovery And Algebraic Equilibrium Proof (2024)0.00
• Co-adaptation Of Algorithmic And Implementational Innovations In Inference-based Deep Reinforcement Learning (2021)0.00
• Distillation Policy Optimization (2023)0.00
• Provably Efficient Off-policy Adversarial Imitation Learning With Convergence Guarantees (2024)0.00