Mean-field Control Based Approximation Of Multi-agent Reinforcement Learning In Presence Of A Non-decomposable Shared Global State

Abstract

Mean Field Control (MFC) is a powerful approximation tool to solve large-scale Multi-Agent Reinforcement Learning (MARL) problems. However, the success of MFC relies on the presumption that given the local states and actions of all the agents, the next (local) states of the agents evolve conditionally independent of each other. Here we demonstrate that even in a MARL setting where agents share a common global state in addition to their local states evolving conditionally independently (thus introducing a correlation between the state transition processes of individual agents), the MFC can still be applied as a good approximation tool. The global state is assumed to be non-decomposable i.e., it cannot be expressed as a collection of local states of the agents. We compute the approximation error as \(\mathcal\{O\}(e)\) where \(e=\frac\{1\}\{\sqrt\{N\}\}\left[\sqrt\{|\mathcal\{X\}|\} +\sqrt\{|\mathcal\{U\}|\}\right]\). The size of the agent population is denoted by the term \(N\), and \(|

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keymondal2023mean

Related papers

• Can Mean Field Control (MFC) Approximate Cooperative Multi Agent Reinforcement Learning (MARL) With Non-uniform Interaction? (2022)0.00
• Mean-field Approximation Of Cooperative Constrained Multi-agent Reinforcement Learning (CMARL) (2022)0.00
• On The Approximation Of Cooperative Heterogeneous Multi-agent Reinforcement Learning (MARL) Using Mean Field Control (MFC) (2021)0.00
• Major-minor Mean Field Multi-agent Reinforcement Learning (2023)0.00
• Efficient Model-based Multi-agent Mean-field Reinforcement Learning (2021)0.00
• Efficient And Scalable Deep Reinforcement Learning For Mean Field Control Games (2024)0.00
• Analysis Of Multiscale Reinforcement Q-learning Algorithms For Mean Field Control Games (2024)0.00
• Mean-field Multi-agent Reinforcement Learning: A Decentralized Network Approach (2021)0.00