Investigating Pre-training Objectives For Generalization In Vision-based Reinforcement Learning

Abstract

Recently, various pre-training methods have been introduced in vision-based Reinforcement Learning (RL). However, their generalization ability remains unclear due to evaluations being limited to in-distribution environments and non-unified experimental setups. To address this, we introduce the Atari Pre-training Benchmark (Atari-PB), which pre-trains a ResNet-50 model on 10 million transitions from 50 Atari games and evaluates it across diverse environment distributions. Our experiments show that pre-training objectives focused on learning task-agnostic features (e.g., identifying objects and understanding temporal dynamics) enhance generalization across different environments. In contrast, objectives focused on learning task-specific knowledge (e.g., identifying agents and fitting reward functions) improve performance in environments similar to the pre-training dataset but not in varied ones. We publicize our codes, datasets, and model checkpoints at https://github.com/dojeon-ai/Atari

Stats

• citations0
• S2 citations—
• github stars11
• HF likes0
• heat score2.16
• arxiv keykim2024investigating

Code

• dojeon-ai/Atari—

Related papers

• Assessing Generalization In Deep Reinforcement Learning (2018)0.00
• The Surprising Ineffectiveness Of Pre-trained Visual Representations For Model-based Reinforcement Learning (2024)0.00
• Generalization And Regularization In DQN (2018)0.00
• Visual Transfer Between Atari Games Using Competitive Reinforcement Learning (2018)7.50
• On The Power Of Pre-training For Generalization In RL: Provable Benefits And Hardness (2022)0.00
• Importance Of Using Appropriate Baselines For Evaluation Of Data-efficiency In Deep Reinforcement Learning For Atari (2020)0.00
• Quantifying Generalization In Reinforcement Learning (2018)0.00
• The Role Of Pretrained Representations For The OOD Generalization Of Reinforcement Learning Agents (2021)0.00