Policy Smoothing For Provably Robust Reinforcement Learning

Abstract

The study of provable adversarial robustness for deep neural networks (DNNs) has mainly focused on static supervised learning tasks such as image classification. However, DNNs have been used extensively in real-world adaptive tasks such as reinforcement learning (RL), making such systems vulnerable to adversarial attacks as well. Prior works in provable robustness in RL seek to certify the behaviour of the victim policy at every time-step against a non-adaptive adversary using methods developed for the static setting. But in the real world, an RL adversary can infer the defense strategy used by the victim agent by observing the states, actions, etc., from previous time-steps and adapt itself to produce stronger attacks in future steps. We present an efficient procedure, designed specifically to defend against an adaptive RL adversary, that can directly certify the total reward without requiring the policy to be robust at each time-step. Our main theoretical contribution is to prove an

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keykumar2021policy

Related papers

• Attacking And Defending Deep Reinforcement Learning Policies (2022)0.00
• Adversary Agnostic Robust Deep Reinforcement Learning (2020)6.77
• Regret-based Defense In Adversarial Reinforcement Learning (2023)0.00
• Robust Reinforcement Learning On State Observations With Learned Optimal Adversary (2021)0.00
• Towards Robust Policy: Enhancing Offline Reinforcement Learning With Adversarial Attacks And Defenses (2024)3.58
• Safe Reinforcement Learning With Dual Robustness (2023)8.60
• Robust Deep Reinforcement Learning Against Adversarial Perturbations On State Observations (2020)0.00
• Online Robust Policy Learning In The Presence Of Unknown Adversaries (2018)0.00