Decentralized Transformers With Centralized Aggregation Are Sample-efficient Multi-agent World Models

Abstract

Learning a world model for model-free Reinforcement Learning (RL) agents can significantly improve the sample efficiency by learning policies in imagination. However, building a world model for Multi-Agent RL (MARL) can be particularly challenging due to the scalability issue in a centralized architecture arising from a large number of agents, and also the non-stationarity issue in a decentralized architecture stemming from the inter-dependency among agents. To address both challenges, we propose a novel world model for MARL that learns decentralized local dynamics for scalability, combined with a centralized representation aggregation from all agents. We cast the dynamics learning as an auto-regressive sequence modeling problem over discrete tokens by leveraging the expressive Transformer architecture, in order to model complex local dynamics across different agents and provide accurate and consistent long-term imaginations. As the first pioneering Transformer-based world model for mu

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyzhang2024decentralized

Related papers

• Transformers Are Sample-efficient World Models (2022)0.00
• MABL: Bi-level Latent-variable World Model For Sample-efficient Multi-agent Reinforcement Learning (2023)0.00
• Learning To Play Atari In A World Of Tokens (2024)0.00
• Updet: Universal Multi-agent Reinforcement Learning Via Policy Decoupling With Transformers (2021)0.00
• STORM: Efficient Stochastic Transformer Based World Models For Reinforcement Learning (2023)4.52
• Object-centric World Models For Causality-aware Reinforcement Learning (2025)0.00
• Transformer-based Scalable Multi-agent Reinforcement Learning For Networked Systems With Long-range Interactions (2025)0.00
• GAWM: Global-aware World Model For Multi-agent Reinforcement Learning (2025)0.00