Qatten: A General Framework For Cooperative Multiagent Reinforcement Learning

Abstract

In many real-world tasks, multiple agents must learn to coordinate with each other given their private observations and limited communication ability. Deep multiagent reinforcement learning (Deep-MARL) algorithms have shown superior performance in such challenging settings. One representative class of work is multiagent value decomposition, which decomposes the global shared multiagent Q-value \(Q_\{tot\}\) into individual Q-values \(Q^\{i\}\) to guide individuals' behaviors, i.e. VDN imposing an additive formation and QMIX adopting a monotonic assumption using an implicit mixing method. However, most of the previous efforts impose certain assumptions between \(Q_\{tot\}\) and \(Q^\{i\}\) and lack theoretical groundings. Besides, they do not explicitly consider the agent-level impact of individuals to the whole system when transforming individual \(Q^\{i\}\)s into \(Q_\{tot\}\). In this paper, we theoretically derive a general formula of \(Q_\{tot\}\) in terms of \(Q^\{i\}\), based on

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyyang2020qatten

Related papers

• Q-value Path Decomposition For Deep Multiagent Reinforcement Learning (2020)0.00
• Understanding Value Decomposition Algorithms In Deep Cooperative Multi-agent Reinforcement Learning (2022)0.00
• Residual Q-networks For Value Function Factorizing In Multi-agent Reinforcement Learning (2022)10.21
• Weighted QMIX: Expanding Monotonic Value Function Factorisation For Deep Multi-agent Reinforcement Learning (2020)0.00
• Monotonic Value Function Factorisation For Deep Multi-agent Reinforcement Learning (2020)0.00
• N\(\text{a}^\text{2}\)q: Neural Attention Additive Model For Interpretable Multi-agent Q-learning (2023)0.00
• QMIX: Monotonic Value Function Factorisation For Deep Multi-agent Reinforcement Learning (2018)0.00
• Mitigating Relative Over-generalization In Multi-agent Reinforcement Learning (2024)0.00