Enhancing Robustness In Deep Reinforcement Learning: A Lyapunov Exponent Approach

Abstract

Deep reinforcement learning agents achieve state-of-the-art performance in a wide range of simulated control tasks. However, successful applications to real-world problems remain limited. One reason for this dichotomy is because the learnt policies are not robust to observation noise or adversarial attacks. In this paper, we investigate the robustness of deep RL policies to a single small state perturbation in deterministic continuous control tasks. We demonstrate that RL policies can be deterministically chaotic, as small perturbations to the system state have a large impact on subsequent state and reward trajectories. This unstable non-linear behaviour has two consequences: first, inaccuracies in sensor readings, or adversarial attacks, can cause significant performance degradation; second, even policies that show robust performance in terms of rewards may have unpredictable behaviour in practice. These two facets of chaos in RL policies drastically restrict the application of deep R

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyyoung2024enhancing

Related papers

• Robust Adversarial Policy Optimization Under Dynamics Uncertainty (2026)0.00
• Robust Reinforcement Learning On State Observations With Learned Optimal Adversary (2021)0.00
• Robust Deep Reinforcement Learning Against Adversarial Perturbations On State Observations (2020)0.00
• Robust Reinforcement Learning: A Case Study In Linear Quadratic Regulation (2020)11.19
• Robust Deep Reinforcement Learning With Adaptive Adversarial Perturbations In Action Space (2024)6.20
• Adversarial Policies: Attacking Deep Reinforcement Learning (2019)0.00
• On The Robustness Of Safe Reinforcement Learning Under Observational Perturbations (2022)0.00
• Robustifying Reinforcement Learning Policies With \(\mathcal{l}_1\) Adaptive Control (2021)0.00