Challenges And Opportunities In Offline Reinforcement Learning From Visual Observations

Abstract

Offline reinforcement learning has shown great promise in leveraging large pre-collected datasets for policy learning, allowing agents to forgo often-expensive online data collection. However, offline reinforcement learning from visual observations with continuous action spaces remains under-explored, with a limited understanding of the key challenges in this complex domain. In this paper, we establish simple baselines for continuous control in the visual domain and introduce a suite of benchmarking tasks for offline reinforcement learning from visual observations designed to better represent the data distributions present in real-world offline RL problems and guided by a set of desiderata for offline RL from visual observations, including robustness to visual distractions and visually identifiable changes in dynamics. Using this suite of benchmarking tasks, we show that simple modifications to two popular vision-based online reinforcement learning algorithms, DreamerV2 and DrQ-v2, suf

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keylu2022challenges

Related papers

• D4RL: Datasets For Deep Data-driven Reinforcement Learning (2020)0.00
• An Optimistic Perspective On Offline Reinforcement Learning (2019)0.00
• Data Valuation For Offline Reinforcement Learning (2022)0.00
• AWAC: Accelerating Online Reinforcement Learning With Offline Datasets (2020)0.00
• Interpretable Performance Analysis Towards Offline Reinforcement Learning: A Dataset Perspective (2021)0.00
• Offline Vs. Online Learning In Model-based RL: Lessons For Data Collection Strategies (2025)0.00
• Bridging The Gap Between Offline And Online Reinforcement Learning Evaluation Methodologies (2022)0.00
• A Dataset Perspective On Offline Reinforcement Learning (2021)0.00