Deep Q-learning: A Robust Control Approach

Abstract

In this paper, we place deep Q-learning into a control-oriented perspective and study its learning dynamics with well-established techniques from robust control. We formulate an uncertain linear time-invariant model by means of the neural tangent kernel to describe learning. We show the instability of learning and analyze the agent's behavior in frequency-domain. Then, we ensure convergence via robust controllers acting as dynamical rewards in the loss function. We synthesize three controllers: state-feedback gain scheduling H2, dynamic Hinf, and constant gain Hinf controllers. Setting up the learning agent with a control-oriented tuning methodology is more transparent and has well-established literature compared to the heuristics in reinforcement learning. In addition, our approach does not use a target network and randomized replay memory. The role of the target network is overtaken by the control input, which also exploits the temporal dependency of samples (opposed to a randomized

Stats

• citations16
• S2 citations—
• github stars0
• HF likes0
• heat score9.23
• arxiv keyvarga2022deep

Related papers

• Deep Q-learning: Theoretical Insights From An Asymptotic Analysis (2020)10.35
• Universal Approximation Theorem Of Deep Q-networks (2025)0.00
• Continuous-time Q-learning For Mean-field Control Problems (2023)0.00
• Actively Learning Reinforcement Learning: A Stochastic Optimal Control Approach (2023)0.00
• Robust Reinforcement Learning: A Case Study In Linear Quadratic Regulation (2020)11.19
• Learning The Linear Quadratic Regulator From Nonlinear Observations (2020)0.00
• A Tour Of Reinforcement Learning: The View From Continuous Control (2018)19.86
• Model-based Reinforcement Learning For Control Under Time-varying Dynamics (2026)0.00