Efficient Adversarial Training Without Attacking: Worst-case-aware Robust Reinforcement Learning

Abstract

Recent studies reveal that a well-trained deep reinforcement learning (RL) policy can be particularly vulnerable to adversarial perturbations on input observations. Therefore, it is crucial to train RL agents that are robust against any attacks with a bounded budget. Existing robust training methods in deep RL either treat correlated steps separately, ignoring the robustness of long-term rewards, or train the agents and RL-based attacker together, doubling the computational burden and sample complexity of the training process. In this work, we propose a strong and efficient robust training framework for RL, named Worst-case-aware Robust RL (WocaR-RL) that directly estimates and optimizes the worst-case reward of a policy under bounded l_p attacks without requiring extra samples for learning an attacker. Experiments on multiple environments show that WocaR-RL achieves state-of-the-art performance under various strong attacks, and obtains significantly higher training efficiency than pri

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyliang2022efficient

Related papers

• Robust Reinforcement Learning On State Observations With Learned Optimal Adversary (2021)0.00
• Robust Deep Reinforcement Learning Through Adversarial Loss (2020)0.00
• On The Robustness Of Safe Reinforcement Learning Under Observational Perturbations (2022)0.00
• Towards Robust Policy: Enhancing Offline Reinforcement Learning With Adversarial Attacks And Defenses (2024)3.58
• Attacking And Defending Deep Reinforcement Learning Policies (2022)0.00
• Combining Pessimism With Optimism For Robust And Efficient Model-based Deep Reinforcement Learning (2021)0.00
• Safe Reinforcement Learning With Dual Robustness (2023)8.60
• Adversary Agnostic Robust Deep Reinforcement Learning (2020)6.77