Causal Mean Field Multi-agent Reinforcement Learning

Abstract

Scalability remains a challenge in multi-agent reinforcement learning and is currently under active research. A framework named mean-field reinforcement learning (MFRL) could alleviate the scalability problem by employing the Mean Field Theory to turn a many-agent problem into a two-agent problem. However, this framework lacks the ability to identify essential interactions under nonstationary environments. Causality contains relatively invariant mechanisms behind interactions, though environments are nonstationary. Therefore, we propose an algorithm called causal mean-field Q-learning (CMFQ) to address the scalability problem. CMFQ is ever more robust toward the change of the number of agents though inheriting the compressed representation of MFRL's action-state space. Firstly, we model the causality behind the decision-making process of MFRL into a structural causal model (SCM). Then the essential degree of each interaction is quantified via intervening on the SCM. Furthermore, we des

Stats

• citations1
• S2 citations—
• github stars0
• HF likes0
• heat score2.26
• arxiv keyma2025causal

Related papers

• Partially Observable Mean Field Reinforcement Learning (2020)0.00
• Mean Field Multi-agent Reinforcement Learning (2018)2.26
• Efficient Model-based Multi-agent Mean-field Reinforcement Learning (2021)0.00
• Breaking The Curse Of Many Agents: Provable Mean Embedding Q-iteration For Mean-field Reinforcement Learning (2020)0.00
• Mean-field Reinforcement Learning Without Synchrony (2026)0.00
• Analysis Of Multiscale Reinforcement Q-learning Algorithms For Mean Field Control Games (2024)0.00
• Model-free Mean-field Reinforcement Learning: Mean-field MDP And Mean-field Q-learning (2019)0.00
• Multi Type Mean Field Reinforcement Learning (2020)0.00