Common Information Based Approximate State Representations In Multi-agent Reinforcement Learning

Abstract

Due to information asymmetry, finding optimal policies for Decentralized Partially Observable Markov Decision Processes (Dec-POMDPs) is hard with the complexity growing doubly exponentially in the horizon length. The challenge increases greatly in the multi-agent reinforcement learning (MARL) setting where the transition probabilities, observation kernel, and reward function are unknown. Here, we develop a general compression framework with approximate common and private state representations, based on which decentralized policies can be constructed. We derive the optimality gap of executing dynamic programming (DP) with the approximate states in terms of the approximation error parameters and the remaining time steps. When the compression is exact (no error), the resulting DP is equivalent to the one in existing work. Our general framework generalizes a number of methods proposed in the literature. The results shed light on designing practically useful deep-MARL network structures und

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keykao2021common

Related papers

• Information State Embedding In Partially Observable Cooperative Multi-agent Reinforcement Learning (2020)0.00
• Probing Dec-pomdp Reasoning In Cooperative MARL (2026)0.00
• Sample-efficient Reinforcement Learning Of Partially Observable Markov Games (2022)0.00
• Remembering The Markov Property In Cooperative MARL (2025)0.00
• Centralized Model And Exploration Policy For Multi-agent RL (2021)0.00
• Macro-action-based Multi-agent/robot Deep Reinforcement Learning Under Partial Observability (2022)5.84
• Breaking The Curse Of Multiagency: Provably Efficient Decentralized Multi-agent RL With Function Approximation (2023)0.00
• Generalizing Multi-step Inverse Models For Representation Learning To Finite-memory Pomdps (2024)0.00