Extrinsicaly Rewarded Soft Q Imitation Learning With Discriminator

Abstract

Imitation learning is often used in addition to reinforcement learning in environments where reward design is difficult or where the reward is sparse, but it is difficult to be able to imitate well in unknown states from a small amount of expert data and sampling data. Supervised learning methods such as Behavioral Cloning do not require sampling data, but usually suffer from distribution shift. The methods based on reinforcement learning, such as inverse reinforcement learning and Generative Adversarial imitation learning (GAIL), can learn from only a few expert data. However, they often need to interact with the environment. Soft Q imitation learning (SQIL) addressed the problems, and it was shown that it could learn efficiently by combining Behavioral Cloning and soft Q-learning with constant rewards. In order to make this algorithm more robust to distribution shift, we propose more efficient and robust algorithm by adding to this method a reward function based on adversarial invers

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyfuruyama2024extrinsicaly

Related papers

• Discriminator Soft Actor Critic Without Extrinsic Rewards (2020)3.58
• Iq-learn: Inverse Soft-q Learning For Imitation (2021)0.00
• Diffail: Diffusion Adversarial Imitation Learning (2023)9.13
• RLIF: Interactive Imitation Learning As Reinforcement Learning (2023)0.00
• Adversarial Soft Advantage Fitting: Imitation Learning Without Policy Optimization (2020)0.00
• Augmented Q Imitation Learning (AQIL) (2020)0.00
• Self-supervised Adversarial Imitation Learning (2023)0.00
• On Discovering Algorithms For Adversarial Imitation Learning (2025)0.00