Distributed Q-learning With State Tracking For Multi-agent Networked Control

Abstract

This paper studies distributed Q-learning for Linear Quadratic Regulator (LQR) in a multi-agent network. The existing results often assume that agents can observe the global system state, which may be infeasible in large-scale systems due to privacy concerns or communication constraints. In this work, we consider a setting with unknown system models and no centralized coordinator. We devise a state tracking (ST) based Q-learning algorithm to design optimal controllers for agents. Specifically, we assume that agents maintain local estimates of the global state based on their local information and communications with neighbors. At each step, every agent updates its local global state estimation, based on which it solves an approximate Q-factor locally through policy iteration. Assuming decaying injected excitation noise during the policy evaluation, we prove that the local estimation converges to the true global state, and establish the convergence of the proposed distributed ST-based Q-

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keywang2020distributed

Related papers

• Natural Actor-critic Converges Globally For Hierarchical Linear Quadratic Regulator (2019)0.00
• Exploiting Inter-agent Coupling Information For Efficient Reinforcement Learning Of Cooperative LQR (2025)0.00
• Least-squares Temporal Difference Learning For The Linear Quadratic Regulator (2017)0.00
• Learning The Linear Quadratic Regulator From Nonlinear Observations (2020)0.00
• Harnessing Data From Clustered LQR Systems: Personalized And Collaborative Policy Optimization (2025)0.00
• Robust Reinforcement Learning: A Case Study In Linear Quadratic Regulation (2020)11.19
• Meta-learning Linear Quadratic Regulators: A Policy Gradient MAML Approach For Model-free LQR (2024)0.00
• Learning Robust Control For LQR Systems With Multiplicative Noise Via Policy Gradient (2019)0.00