Smaller World Models For Reinforcement Learning

Abstract

Sample efficiency remains a fundamental issue of reinforcement learning. Model-based algorithms try to make better use of data by simulating the environment with a model. We propose a new neural network architecture for world models based on a vector quantized-variational autoencoder (VQ-VAE) to encode observations and a convolutional LSTM to predict the next embedding indices. A model-free PPO agent is trained purely on simulated experience from the world model. We adopt the setup introduced by Kaiser et al. (2020), which only allows 100K interactions with the real environment. We apply our method on 36 Atari environments and show that we reach comparable performance to their SimPLe algorithm, while our model is significantly smaller.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyrobine2020smaller

Related papers

• Recurrent World Models Facilitate Policy Evolution (2018)0.00
• The Effectiveness Of World Models For Continual Reinforcement Learning (2022)0.00
• Transformers Are Sample-efficient World Models (2022)0.00
• STORM: Efficient Stochastic Transformer Based World Models For Reinforcement Learning (2023)4.52
• World Models As An Intermediary Between Agents And The Real World (2026)0.00
• Object-centric World Models For Causality-aware Reinforcement Learning (2025)0.00
• Continual Learning Using World Models For Pseudo-rehearsal (2019)0.00
• Model-based Reinforcement Learning For Atari (2019)0.00