Belief States For Cooperative Multi-agent Reinforcement Learning Under Partial Observability

Abstract

Reinforcement learning in partially observable environments is typically challenging, as it requires agents to learn an estimate of the underlying system state. These challenges are exacerbated in multi-agent settings, where agents learn simultaneously and influence the underlying state as well as each others' observations. We propose the use of learned beliefs on the underlying state of the system to overcome these challenges and enable reinforcement learning with fully decentralized training and execution. Our approach leverages state information to pre-train a probabilistic belief model in a self-supervised fashion. The resulting belief states, which capture both inferred state information as well as uncertainty over this information, are then used in a state-based reinforcement learning algorithm to create an end-to-end model for cooperative multi-agent reinforcement learning under partial observability. By separating the belief and reinforcement learning tasks, we are able to sign

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keypritz2025belief

Related papers

• Deep Decentralized Multi-task Multi-agent Reinforcement Learning Under Partial Observability (2017)0.00
• Information State Embedding In Partially Observable Cooperative Multi-agent Reinforcement Learning (2020)0.00
• Multi-agent Off-policy Actor-critic Reinforcement Learning For Partially Observable Environments (2024)2.26
• Recurrent Networks, Hidden States And Beliefs In Partially Observable Environments (2022)0.00
• Reinforcement Learning Under Partial Observability Guided By Learned Environment Models (2022)6.34
• Agent Modelling Under Partial Observability For Deep Reinforcement Learning (2020)0.00
• Policy Evaluation In Decentralized Pomdps With Belief Sharing (2023)0.00
• Unbiased Asymmetric Reinforcement Learning Under Partial Observability (2021)2.26