QGNN: Value Function Factorisation With Graph Neural Networks

Abstract

In multi-agent reinforcement learning, the use of a global objective is a powerful tool for incentivising cooperation. Unfortunately, it is not sample-efficient to train individual agents with a global reward, because it does not necessarily correlate with an agent's individual actions. This problem can be solved by factorising the global value function into local value functions. Early work in this domain performed factorisation by conditioning local value functions purely on local information. Recently, it has been shown that providing both local information and an encoding of the global state can promote cooperative behaviour. In this paper we propose QGNN, the first value factorisation method to use a graph neural network (GNN) based model. The multi-layer message passing architecture of QGNN provides more representational complexity than models in prior work, allowing it to produce a more effective factorisation. QGNN also introduces a permutation invariant mixer which is able to

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keykortvelesy2022qgnn

Related papers

• Concaveq: Non-monotonic Value Function Factorization Via Concave Representations In Deep Multi-agent Reinforcement Learning (2023)5.84
• Residual Q-networks For Value Function Factorizing In Multi-agent Reinforcement Learning (2022)10.21
• Qfree: A Universal Value Function Factorization For Multi-agent Reinforcement Learning (2023)0.00
• Towards Understanding Cooperative Multi-agent Q-learning With Value Factorization (2020)0.00
• Monotonic Value Function Factorisation For Deep Multi-agent Reinforcement Learning (2020)0.00
• NQMIX: Non-monotonic Value Function Factorization For Deep Multi-agent Reinforcement Learning (2021)0.00
• QMIX: Monotonic Value Function Factorisation For Deep Multi-agent Reinforcement Learning (2018)0.00
• Factored Value Functions For Graph-based Multi-agent Reinforcement Learning (2026)0.00