AWAC: Accelerating Online Reinforcement Learning With Offline Datasets

Abstract

Reinforcement learning (RL) provides an appealing formalism for learning control policies from experience. However, the classic active formulation of RL necessitates a lengthy active exploration process for each behavior, making it difficult to apply in real-world settings such as robotic control. If we can instead allow RL algorithms to effectively use previously collected data to aid the online learning process, such applications could be made substantially more practical: the prior data would provide a starting point that mitigates challenges due to exploration and sample complexity, while the online training enables the agent to perfect the desired skill. Such prior data could either constitute expert demonstrations or sub-optimal prior data that illustrates potentially useful transitions. While a number of prior methods have either used optimal demonstrations to bootstrap RL, or have used sub-optimal data to train purely offline, it remains exceptionally difficult to train a polic

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keynair2020awac

Related papers

• Active Advantage-aligned Online Reinforcement Learning With Offline Data (2025)0.00
• A Workflow For Offline Model-free Robotic Reinforcement Learning (2021)0.00
• D4RL: Datasets For Deep Data-driven Reinforcement Learning (2020)0.00
• An Optimistic Perspective On Offline Reinforcement Learning (2019)0.00
• Using Offline Data To Speed Up Reinforcement Learning In Procedurally Generated Environments (2023)6.77
• Towards Data-driven Offline Simulations For Online Reinforcement Learning (2022)0.00
• Don't Change The Algorithm, Change The Data: Exploratory Data For Offline Reinforcement Learning (2022)0.00
• Leveraging Offline Data In Online Reinforcement Learning (2022)0.00