Online Reinforcement Learning Control By Direct Heuristic Dynamic Programming: From Time-driven To Event-driven

Abstract

In this paper time-driven learning refers to the machine learning method that updates parameters in a prediction model continuously as new data arrives. Among existing approximate dynamic programming (ADP) and reinforcement learning (RL) algorithms, the direct heuristic dynamic programming (dHDP) has been shown an effective tool as demonstrated in solving several complex learning control problems. It continuously updates the control policy and the critic as system states continuously evolve. It is therefore desirable to prevent the time-driven dHDP from updating due to insignificant system event such as noise. Toward this goal, we propose a new event-driven dHDP. By constructing a Lyapunov function candidate, we prove the uniformly ultimately boundedness (UUB) of the system states and the weights in the critic and the control policy networks. Consequently we show the approximate control and cost-to-go function approaching Bellman optimality within a finite bound. We also illustrate how

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyzhao2020online

Related papers

• Hyperl: Hypernetwork-based Reinforcement Learning For Control Of Parametrized Dynamical Systems (2025)0.00
• Model Predictive Control And Reinforcement Learning: A Unified Framework Based On Dynamic Programming (2024)10.61
• Model-based Reinforcement Learning For Control Under Time-varying Dynamics (2026)0.00
• Application Of Deep Reinforcement Learning To Event-triggered Control For Networked Artificial Pancreas Systems (2026)0.00
• A General Markov Decision Process Framework For Directly Learning Optimal Control Policies (2019)0.00
• RLOC: Neurobiologically Inspired Hierarchical Reinforcement Learning Algorithm For Continuous Control Of Nonlinear Dynamical Systems (2019)0.00
• Actively Learning Reinforcement Learning: A Stochastic Optimal Control Approach (2023)0.00
• Adaptive Dynamic Programming For Model-free Tracking Of Trajectories With Time-varying Parameters (2019)9.59