On Minimizing Adversarial Counterfactual Error In Adversarial RL

Abstract

Deep Reinforcement Learning (DRL) policies are highly susceptible to adversarial noise in observations, which poses significant risks in safety-critical scenarios. The challenge inherent to adversarial perturbations is that by altering the information observed by the agent, the state becomes only partially observable. Existing approaches address this by either enforcing consistent actions across nearby states or maximizing the worst-case value within adversarially perturbed observations. However, the former suffers from performance degradation when attacks succeed, while the latter tends to be overly conservative, leading to suboptimal performance in benign settings. We hypothesize that these limitations stem from their failing to account for partial observability directly. To this end, we introduce a novel objective called Adversarial Counterfactual Error (ACoE), defined on the beliefs about the true state and balancing value optimization with robustness. To make ACoE scalable in mode

Stats

• citations0
• S2 citations—
• github stars1
• HF likes0
• heat score0.60
• arxiv keybelaire2024on

Related papers

• Regret-based Defense In Adversarial Reinforcement Learning (2023)0.00
• Robust Reinforcement Learning On State Observations With Learned Optimal Adversary (2021)0.00
• Robust Deep Reinforcement Learning Against Adversarial Perturbations On State Observations (2020)0.00
• Adversarial Policies: Attacking Deep Reinforcement Learning (2019)0.00
• Adversary Agnostic Robust Deep Reinforcement Learning (2020)6.77
• Attacking And Defending Deep Reinforcement Learning Policies (2022)0.00
• Acrel: Adversarial Conditional Value-at-risk Reinforcement Learning (2021)0.00
• Robust Deep Reinforcement Learning With Adaptive Adversarial Perturbations In Action Space (2024)6.20