Drama: Mamba-enabled Model-based Reinforcement Learning Is Sample And Parameter Efficient

Abstract

Model-based reinforcement learning (RL) offers a solution to the data inefficiency that plagues most model-free RL algorithms. However, learning a robust world model often requires complex and deep architectures, which are computationally expensive and challenging to train. Within the world model, sequence models play a critical role in accurate predictions, and various architectures have been explored, each with its own challenges. Currently, recurrent neural network (RNN)-based world models struggle with vanishing gradients and capturing long-term dependencies. Transformers, on the other hand, suffer from the quadratic memory and computational complexity of self-attention mechanisms, scaling as \(O(n^2)\), where \(n\) is the sequence length. To address these challenges, we propose a state space model (SSM)-based world model, Drama, specifically leveraging Mamba, that achieves \(O(n)\) memory and computational complexity while effectively capturing long-term dependencies and enablin

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keywang2024drama

Related papers

• Decision Mamba: A Multi-grained State Space Model With Self-evolution Regularization For Offline RL (2024)0.00
• Decentralized Transformers With Centralized Aggregation Are Sample-efficient Multi-agent World Models (2024)0.00
• Decision Mamba: Reinforcement Learning Via Sequence Modeling With Selective State Spaces (2024)0.00
• MABL: Bi-level Latent-variable World Model For Sample-efficient Multi-agent Reinforcement Learning (2023)0.00
• STORM: Efficient Stochastic Transformer Based World Models For Reinforcement Learning (2023)4.52
• VMAV-C: A Deep Attention-based Reinforcement Learning Algorithm For Model-based Control (2018)0.00
• Harmonydream: Task Harmonization Inside World Models (2023)3.46
• Smaller World Models For Reinforcement Learning (2020)0.00