Lyapunov-based Reinforcement Learning State Estimator

Abstract

In this paper, we consider the state estimation problem for nonlinear stochastic discrete-time systems. We combine Lyapunov's method in control theory and deep reinforcement learning to design the state estimator. We theoretically prove the convergence of the bounded estimate error solely using the data simulated from the model. An actor-critic reinforcement learning algorithm is proposed to learn the state estimator approximated by a deep neural network. The convergence of the algorithm is analysed. The proposed Lyapunov-based reinforcement learning state estimator is compared with a number of existing nonlinear filtering methods through Monte Carlo simulations, showing its advantage in terms of estimate convergence even under some system uncertainties such as covariance shift in system noise and randomly missing measurements. To the best of our knowledge, this is the first reinforcement learning based nonlinear state estimator with bounded estimate error performance guarantee.

Stats

• citations11
• S2 citations—
• github stars0
• HF likes0
• heat score8.09
• arxiv keyhu2020lyapunov

Related papers

• Enhancing Robustness In Deep Reinforcement Learning: A Lyapunov Exponent Approach (2024)0.00
• Adaptive Input Estimation In Linear Dynamical Systems With Applications To Learning-from-observations (2018)0.00
• Stable Reinforcement Learning With Unbounded State Space (2020)0.00
• SBEED: Convergent Reinforcement Learning With Nonlinear Function Approximation (2017)0.00
• An Efficient, Expressive And Local Minima-free Method For Learning Controlled Dynamical Systems (2017)2.26
• Sample Complexity Of Estimating The Policy Gradient For Nearly Deterministic Dynamical Systems (2019)0.00
• Least-squares Temporal Difference Learning For The Linear Quadratic Regulator (2017)0.00
• Distributed Q-learning With State Tracking For Multi-agent Networked Control (2020)0.00