Policy Resilience To Environment Poisoning Attacks On Reinforcement Learning

Abstract

This paper investigates policy resilience to training-environment poisoning attacks on reinforcement learning (RL) policies, with the goal of recovering the deployment performance of a poisoned RL policy. Due to the fact that the policy resilience is an add-on concern to RL algorithms, it should be resource-efficient, time-conserving, and widely applicable without compromising the performance of RL algorithms. This paper proposes such a policy-resilience mechanism based on an idea of knowledge sharing. We summarize the policy resilience as three stages: preparation, diagnosis, recovery. Specifically, we design the mechanism as a federated architecture coupled with a meta-learning manner, pursuing an efficient extraction and sharing of the environment knowledge. With the shared knowledge, a poisoned agent can quickly identify the deployment condition and accordingly recover its policy performance. We empirically evaluate the resilience mechanism for both model-based and model-free RL al

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyxu2023policy

Related papers

• Policy Teaching In Reinforcement Learning Via Environment Poisoning Attacks (2020)0.00
• Vulnerability-aware Poisoning Mechanism For Online RL With Unknown Dynamics (2020)0.00
• Local Environment Poisoning Attacks On Federated Reinforcement Learning (2023)0.00
• Online Poisoning Attack Against Reinforcement Learning Under Black-box Environments (2024)0.00
• COPA: Certifying Robust Policies For Offline Reinforcement Learning Against Poisoning Attacks (2022)0.00
• Reward Poisoning In Reinforcement Learning: Attacks Against Unknown Learners In Unknown Environments (2021)0.00
• Efficient Reward Poisoning Attacks On Online Deep Reinforcement Learning (2022)0.00
• Understanding The Limits Of Poisoning Attacks In Episodic Reinforcement Learning (2022)3.58