Online Adaptation For Enhancing Imitation Learning Policies

Abstract

Imitation learning enables autonomous agents to learn from human examples, without the need for a reward signal. Still, if the provided dataset does not encapsulate the task correctly, or when the task is too complex to be modeled, such agents fail to reproduce the expert policy. We propose to recover from these failures through online adaptation. Our approach combines the action proposal coming from a pre-trained policy with relevant experience recorded by an expert. The combination results in an adapted action that closely follows the expert. Our experiments show that an adapted agent performs better than its pure imitation learning counterpart. Notably, adapted agents can achieve reasonable performance even when the base, non-adapted policy catastrophically fails.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keymalato2024online

Related papers

• Mimicking Better By Matching The Approximate Action Distribution (2023)0.00
• Minimax Optimal Online Imitation Learning Via Replay Estimation (2022)0.00
• Imitating Opponent To Win: Adversarial Policy Imitation Learning In Two-player Competitive Games (2022)0.00
• Explaining Fast Improvement In Online Imitation Learning (2020)0.00
• Adversarial Imitation Learning Via Random Search (2020)7.16
• Efficient Offline Reinforcement Learning: First Imitate, Then Improve (2024)1.91
• Accelerating Imitation Learning With Predictive Models (2018)0.00
• A Dual Approach To Imitation Learning From Observations With Offline Datasets (2024)0.00