Online Robust Policy Learning In The Presence Of Unknown Adversaries

Abstract

The growing prospect of deep reinforcement learning (DRL) being used in cyber-physical systems has raised concerns around safety and robustness of autonomous agents. Recent work on generating adversarial attacks have shown that it is computationally feasible for a bad actor to fool a DRL policy into behaving sub optimally. Although certain adversarial attacks with specific attack models have been addressed, most studies are only interested in off-line optimization in the data space (e.g., example fitting, distillation). This paper introduces a Meta-Learned Advantage Hierarchy (MLAH) framework that is attack model-agnostic and more suited to reinforcement learning, via handling the attacks in the decision space (as opposed to data space) and directly mitigating learned bias introduced by the adversary. In MLAH, we learn separate sub-policies (nominal and adversarial) in an online manner, as guided by a supervisory master agent that detects the presence of the adversary by leveraging the

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyhavens2018online

Related papers

• Learning To Cope With Adversarial Attacks (2019)0.00
• Towards Robust Policy: Enhancing Offline Reinforcement Learning With Adversarial Attacks And Defenses (2024)3.58
• Adversarial Policies: Attacking Deep Reinforcement Learning (2019)0.00
• Neutral Agent-based Adversarial Policy Learning Against Deep Reinforcement Learning In Multi-party Open Systems (2025)0.00
• Attacking And Defending Deep Reinforcement Learning Policies (2022)0.00
• Adversary Agnostic Robust Deep Reinforcement Learning (2020)6.77
• Robust Reinforcement Learning On State Observations With Learned Optimal Adversary (2021)0.00
• Robust Model-based Reinforcement Learning With An Adversarial Auxiliary Model (2024)0.00