Information State Embedding In Partially Observable Cooperative Multi-agent Reinforcement Learning

Abstract

Multi-agent reinforcement learning (MARL) under partial observability has long been considered challenging, primarily due to the requirement for each agent to maintain a belief over all other agents' local histories -- a domain that generally grows exponentially over time. In this work, we investigate a partially observable MARL problem in which agents are cooperative. To enable the development of tractable algorithms, we introduce the concept of an information state embedding that serves to compress agents' histories. We quantify how the compression error influences the resulting value functions for decentralized control. Furthermore, we propose an instance of the embedding based on recurrent neural networks (RNNs). The embedding is then used as an approximate information state, and can be fed into any MARL algorithm. The proposed embed-then-learn pipeline opens the black-box of existing (partially observable) MARL algorithms, allowing us to establish some theoretical guarantees (erro

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keymao2020information

Related papers

• Belief States For Cooperative Multi-agent Reinforcement Learning Under Partial Observability (2025)0.00
• Sample-efficient Reinforcement Learning Of Partially Observable Markov Games (2022)0.00
• Common Information Based Approximate State Representations In Multi-agent Reinforcement Learning (2021)0.00
• Boosting Value Decomposition Via Unit-wise Attentive State Representation For Cooperative Multi-agent Reinforcement Learning (2023)0.00
• Remembering The Markov Property In Cooperative MARL (2025)0.00
• Cooperative Multi-agent Reinforcement Learning With Partial Observations (2020)10.35
• Macro-action-based Multi-agent/robot Deep Reinforcement Learning Under Partial Observability (2022)5.84
• S2RL: Do We Really Need To Perceive All States In Deep Multi-agent Reinforcement Learning? (2022)0.00