Critic As Lyapunov Function (CALF): A Model-free, Stability-ensuring Agent
2024 · Pavel Osinenko, Grigory Yaremenko, Roman Zashchitin, et al.
Abstract
This work presents and showcases a novel reinforcement learning agent called Critic As Lyapunov Function (CALF) which is model-free and ensures online environment, in other words, dynamical system stabilization. Online means that in each learning episode, the said environment is stabilized. This, as demonstrated in a case study with a mobile robot simulator, greatly improves the overall learning performance. The base actor-critic scheme of CALF is analogous to SARSA. The latter did not show any success in reaching the target in our studies. However, a modified version thereof, called SARSA-m here, did succeed in some learning scenarios. Still, CALF greatly outperformed the said approach. CALF was also demonstrated to improve a nominal stabilizer provided to it. In summary, the presented agent may be considered a viable approach to fusing classical control with reinforcement learning. Its concurrent approaches are mostly either offline or model-based, like, for instance, those that fuse
Authors
(none)
Tags
Stats
Related papers
- Adviser-actor-critic: Eliminating Steady-state Error In Reinforcement Learning Control (2025)0.00
- Langevin Soft Actor-critic: Efficient Exploration Through Uncertainty-driven Critic Learning (2025)0.00
- Natural Actor-critic Converges Globally For Hierarchical Linear Quadratic Regulator (2019)0.00
- Visualizing Critic Match Loss Landscapes For Interpretation Of Online Reinforcement Learning Control Algorithms (2026)0.00
- Local Advantage Actor-critic For Robust Multi-agent Deep Reinforcement Learning (2021)7.81
- MSACL: Multi-step Actor-critic Learning With Lyapunov Certificates For Exponentially Stabilizing Control (2025)0.00
- Context-based Soft Actor Critic For Environments With Non-stationary Dynamics (2021)0.00
- Functional Critics Are Essential For Actor-critic: From Off-policy Stability To Efficient Exploration (2025)0.00