Offline Robot Reinforcement Learning With Uncertainty-guided Human Expert Sampling

Abstract

Recent advances in batch (offline) reinforcement learning have shown promising results in learning from available offline data and proved offline reinforcement learning to be an essential toolkit in learning control policies in a model-free setting. An offline reinforcement learning algorithm applied to a dataset collected by a suboptimal non-learning-based algorithm can result in a policy that outperforms the behavior agent used to collect the data. Such a scenario is frequent in robotics, where existing automation is collecting operational data. Although offline learning techniques can learn from data generated by a sub-optimal behavior agent, there is still an opportunity to improve the sample complexity of existing offline reinforcement learning algorithms by strategically introducing human demonstration data into the training process. To this end, we propose a novel approach that uses uncertainty estimation to trigger the injection of human demonstration data and guide policy trai

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keykumar2022offline

Related papers

• AWAC: Accelerating Online Reinforcement Learning With Offline Datasets (2020)0.00
• Expert-supervised Reinforcement Learning For Offline Policy Learning And Evaluation (2020)0.00
• User-interactive Offline Reinforcement Learning (2022)0.00
• Revisiting Design Choices In Offline Model-based Reinforcement Learning (2021)6.34
• Guided Online Distillation: Promoting Safe Reinforcement Learning By Offline Demonstration (2023)4.52
• Towards Robust Offline-to-online Reinforcement Learning Via Uncertainty And Smoothness (2023)5.24
• Uncertainty-based Offline Reinforcement Learning With Diversified Q-ensemble (2021)0.00
• A Workflow For Offline Model-free Robotic Reinforcement Learning (2021)0.00