Robustifying Reinforcement Learning Policies With \(\mathcal{l}_1\) Adaptive Control

Abstract

A reinforcement learning (RL) policy trained in a nominal environment could fail in a new/perturbed environment due to the existence of dynamic variations. Existing robust methods try to obtain a fixed policy for all envisioned dynamic variation scenarios through robust or adversarial training. These methods could lead to conservative performance due to emphasis on the worst case, and often involve tedious modifications to the training environment. We propose an approach to robustifying a pre-trained non-robust RL policy with \(\mathcal\{L\}_1\) adaptive control. Leveraging the capability of an \(\mathcal\{L\}_1\) control law in the fast estimation of and active compensation for dynamic variations, our approach can significantly improve the robustness of an RL policy trained in a standard (i.e., non-robust) way, either in a simulator or in the real world. Numerical experiments are provided to validate the efficacy of the proposed approach.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keycheng2021robustifying

Related papers

• Robust Model Based Reinforcement Learning Using \(\mathcal{l}_1\) Adaptive Control (2024)0.00
• Enhancing Robustness In Deep Reinforcement Learning: A Lyapunov Exponent Approach (2024)0.00
• Robust Adversarial Policy Optimization Under Dynamics Uncertainty (2026)0.00
• Improve Robustness Of Reinforcement Learning Against Observation Perturbations Via \(l_\infty\) Lipschitz Policy Networks (2023)5.24
• Safe Reinforcement Learning With Dual Robustness (2023)8.60
• Robust Deep Reinforcement Learning With Adaptive Adversarial Perturbations In Action Space (2024)6.20
• Reinforcement Learning With Adaptive Regularization For Safe Control Of Critical Systems (2024)0.00
• Robust Reinforcement Learning: A Case Study In Linear Quadratic Regulation (2020)11.19