ARC - Actor Residual Critic For Adversarial Imitation Learning

Abstract

Adversarial Imitation Learning (AIL) is a class of popular state-of-the-art Imitation Learning algorithms commonly used in robotics. In AIL, an artificial adversary's misclassification is used as a reward signal that is optimized by any standard Reinforcement Learning (RL) algorithm. Unlike most RL settings, the reward in AIL is \(differentiable\) but current model-free RL algorithms do not make use of this property to train a policy. The reward is AIL is also shaped since it comes from an adversary. We leverage the differentiability property of the shaped AIL reward function and formulate a class of Actor Residual Critic (ARC) RL algorithms. ARC algorithms draw a parallel to the standard Actor-Critic (AC) algorithms in RL literature and uses a residual critic, \(C\) function (instead of the standard \(Q\) function) to approximate only the discounted future return (excluding the immediate reward). ARC algorithms have similar convergence properties as the standard AC algorithms with the

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keydeka2022arc

Related papers

• On Discovering Algorithms For Adversarial Imitation Learning (2025)0.00
• Rethinking Adversarial Inverse Reinforcement Learning: Policy Imitation, Transferable Reward Recovery And Algebraic Equilibrium Proof (2024)0.00
• Recursive Least Squares Advantage Actor-critic Algorithms (2022)0.00
• Discriminator-actor-critic: Addressing Sample Inefficiency And Reward Bias In Adversarial Imitation Learning (2018)0.00
• Scalable Multi-agent Inverse Reinforcement Learning Via Actor-attention-critic (2020)0.00
• Diffail: Diffusion Adversarial Imitation Learning (2023)9.13
• Adviser-actor-critic: Eliminating Steady-state Error In Reinforcement Learning Control (2025)0.00
• Discriminator Soft Actor Critic Without Extrinsic Rewards (2020)3.58