Exploiting Inter-agent Coupling Information For Efficient Reinforcement Learning Of Cooperative LQR

Abstract

Developing scalable and efficient reinforcement learning algorithms for cooperative multi-agent control has received significant attention over the past years. Existing literature has proposed inexact decompositions of local Q-functions based on empirical information structures between the agents. In this paper, we exploit inter-agent coupling information and propose a systematic approach to exactly decompose the local Q-function of each agent. We develop an approximate least square policy iteration algorithm based on the proposed decomposition and identify two architectures to learn the local Q-function for each agent. We establish that the worst-case sample complexity of the decomposition is equal to the centralized case and derive necessary and sufficient graphical conditions on the inter-agent couplings to achieve better sample efficiency. We demonstrate the improved sample efficiency and computational efficiency on numerical examples.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keysyed2025exploiting

Related papers

• Locality Matters: A Scalable Value Decomposition Approach For Cooperative Multi-agent Reinforcement Learning (2021)0.00
• Mitigating Relative Over-generalization In Multi-agent Reinforcement Learning (2024)0.00
• Distributed Q-learning With State Tracking For Multi-agent Networked Control (2020)0.00
• Harnessing Data From Clustered LQR Systems: Personalized And Collaborative Policy Optimization (2025)0.00
• Provably Efficient Cooperative Multi-agent Reinforcement Learning With Function Approximation (2021)0.00
• Fully Decentralized Cooperative Multi-agent Reinforcement Learning: A Survey (2024)0.00
• Optimization For Reinforcement Learning: From Single Agent To Cooperative Agents (2019)14.62
• MA2QL: A Minimalist Approach To Fully Decentralized Multi-agent Reinforcement Learning (2022)0.00