A Workflow For Offline Model-free Robotic Reinforcement Learning

Abstract

Offline reinforcement learning (RL) enables learning control policies by utilizing only prior experience, without any online interaction. This can allow robots to acquire generalizable skills from large and diverse datasets, without any costly or unsafe online data collection. Despite recent algorithmic advances in offline RL, applying these methods to real-world problems has proven challenging. Although offline RL methods can learn from prior data, there is no clear and well-understood process for making various design choices, from model architecture to algorithm hyperparameters, without actually evaluating the learned policies online. In this paper, our aim is to develop a practical workflow for using offline RL analogous to the relatively well-understood workflows for supervised learning problems. To this end, we devise a set of metrics and conditions that can be tracked over the course of offline training, and can inform the practitioner about how the algorithm and model architect

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keykumar2021a

Related papers

• AWAC: Accelerating Online Reinforcement Learning With Offline Datasets (2020)0.00
• Morel : Model-based Offline Reinforcement Learning (2020)0.00
• Policy-driven World Model Adaptation For Robust Offline Model-based Reinforcement Learning (2025)0.00
• Expressive Value Learning For Scalable Offline Reinforcement Learning (2025)0.00
• Towards Data-driven Offline Simulations For Online Reinforcement Learning (2022)0.00
• Revisiting Design Choices In Offline Model-based Reinforcement Learning (2021)6.34
• Overcoming Model Bias For Robust Offline Deep Reinforcement Learning (2020)11.58
• Deployment-efficient Reinforcement Learning Via Model-based Offline Optimization (2020)0.00