Towards Robust Policy: Enhancing Offline Reinforcement Learning With Adversarial Attacks And Defenses

Abstract

Offline reinforcement learning (RL) addresses the challenge of expensive and high-risk data exploration inherent in RL by pre-training policies on vast amounts of offline data, enabling direct deployment or fine-tuning in real-world environments. However, this training paradigm can compromise policy robustness, leading to degraded performance in practical conditions due to observation perturbations or intentional attacks. While adversarial attacks and defenses have been extensively studied in deep learning, their application in offline RL is limited. This paper proposes a framework to enhance the robustness of offline RL models by leveraging advanced adversarial attacks and defenses. The framework attacks the actor and critic components by perturbing observations during training and using adversarial defenses as regularization to enhance the learned policy. Four attacks and two defenses are introduced and evaluated on the D4RL benchmark. The results show the vulnerability of both the a

Stats

• citations2
• S2 citations—
• github stars0
• HF likes0
• heat score3.58
• arxiv keynguyen2024towards

Related papers

• Online Robust Policy Learning In The Presence Of Unknown Adversaries (2018)0.00
• Attacking And Defending Deep Reinforcement Learning Policies (2022)0.00
• Robust Reinforcement Learning On State Observations With Learned Optimal Adversary (2021)0.00
• Adversarial Policies: Attacking Deep Reinforcement Learning (2019)0.00
• Robust Deep Reinforcement Learning Through Adversarial Attacks And Training : A Survey (2024)0.00
• Real-time Adversarial Perturbations Against Deep Reinforcement Learning Policies: Attacks And Defenses (2021)0.00
• Efficient Adversarial Training Without Attacking: Worst-case-aware Robust Reinforcement Learning (2022)0.00
• Optimal Attack And Defense For Reinforcement Learning (2023)6.34