Iq-learn: Inverse Soft-q Learning For Imitation

Abstract

In many sequential decision-making problems (e.g., robotics control, game playing, sequential prediction), human or expert data is available containing useful information about the task. However, imitation learning (IL) from a small amount of expert data can be challenging in high-dimensional environments with complex dynamics. Behavioral cloning is a simple method that is widely used due to its simplicity of implementation and stable convergence but doesn't utilize any information involving the environment's dynamics. Many existing methods that exploit dynamics information are difficult to train in practice due to an adversarial optimization process over reward and policy approximators or biased, high variance gradient estimators. We introduce a method for dynamics-aware IL which avoids adversarial training by learning a single Q-function, implicitly representing both reward and policy. On standard benchmarks, the implicitly learned rewards show a high positive correlation with the gr

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keygarg2021iq

Related papers

• Extrinsicaly Rewarded Soft Q Imitation Learning With Discriminator (2024)0.00
• Quantum Imitation Learning (2023)4.52
• Augmented Q Imitation Learning (AQIL) (2020)0.00
• Discriminator Soft Actor Critic Without Extrinsic Rewards (2020)3.58
• Multi-intention Inverse Q-learning For Interpretable Behavior Representation (2023)0.00
• State-only Imitation With Transition Dynamics Mismatch (2020)0.00
• Is Behavior Cloning All You Need? Understanding Horizon In Imitation Learning (2024)0.00
• Imitation Learning Via Differentiable Physics (2022)0.00