Harnessing Data From Clustered LQR Systems: Personalized And Collaborative Policy Optimization
2025 · Vinay Kanakeri, Shivam Bajaj, Ashwin Verma, et al.
Abstract
It is known that reinforcement learning (RL) is data-hungry. To improve sample-efficiency of RL, it has been proposed that the learning algorithm utilize data from 'approximately similar' processes. However, since the process models are unknown, identifying which other processes are similar poses a challenge. In this work, we study this problem in the context of the benchmark Linear Quadratic Regulator (LQR) setting. Specifically, we consider a setting with multiple agents, each corresponding to a copy of a linear process to be controlled. The agents' local processes can be partitioned into clusters based on similarities in dynamics and tasks. Combining ideas from sequential elimination and zeroth-order policy optimization, we propose a new algorithm that performs simultaneous clustering and learning to output a personalized policy (controller) for each cluster. Under a suitable notion of cluster separation that captures differences in closed-loop performance across systems, we prove t
Authors
(none)
Tags
Stats
Related papers
- Some Remarks On Gradient Dominance And LQR Policy Optimization (2025)0.00
- Learning Robust Control For LQR Systems With Multiplicative Noise Via Policy Gradient (2019)0.00
- Exploiting Inter-agent Coupling Information For Efficient Reinforcement Learning Of Cooperative LQR (2025)0.00
- Meta-learning Linear Quadratic Regulators: A Policy Gradient MAML Approach For Model-free LQR (2024)0.00
- On The Optimization Landscape Of Dynamic Output Feedback: A Case Study For Linear Quadratic Regulator (2022)4.52
- The Gap Between Model-based And Model-free Methods On The Linear Quadratic Regulator: An Asymptotic Viewpoint (2018)0.00
- Robust Reinforcement Learning: A Case Study In Linear Quadratic Regulation (2020)11.19
- Exploration-driven Policy Optimization In RLHF: Theoretical Insights On Efficient Data Utilization (2024)0.00