High-order Interactions Modeling For Interpretable Multi-agent Q-learning

Abstract

The ability to model interactions among agents is crucial for effective coordination and understanding their cooperation mechanisms in multi-agent reinforcement learning (MARL). However, previous efforts to model high-order interactions have been primarily hindered by the combinatorial explosion or the opaque nature of their black-box network structures. In this paper, we propose a novel value decomposition framework, called Continued Fraction Q-Learning (QCoFr), which can flexibly capture arbitrary-order agent interactions with only linear complexity \(\mathcal\{O\}\left(\{n\}\right)\) in the number of agents, thus avoiding the combinatorial explosion when modeling rich cooperation. Furthermore, we introduce the variational information bottleneck to extract latent information for estimating credits. This latent information helps agents filter out noisy interactions, thereby significantly enhancing both cooperation and interpretability. Extensive experiments demonstrate that QCoFr not

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyxu2025high

Related papers

• Qatten: A General Framework For Cooperative Multiagent Reinforcement Learning (2020)0.00
• Modeling The Interaction Between Agents In Cooperative Multi-agent Reinforcement Learning (2021)0.00
• N\(\text{a}^\text{2}\)q: Neural Attention Additive Model For Interpretable Multi-agent Q-learning (2023)0.00
• Towards Understanding Cooperative Multi-agent Q-learning With Value Factorization (2020)0.00
• Residual Q-networks For Value Function Factorizing In Multi-agent Reinforcement Learning (2022)10.21
• Q-value Path Decomposition For Deep Multiagent Reinforcement Learning (2020)0.00
• Factorized Q-learning For Large-scale Multi-agent Systems (2018)11.58
• Mixed Q-functionals: Advancing Value-based Methods In Cooperative MARL With Continuous Action Domains (2024)0.00