Policycleanse: Backdoor Detection And Mitigation In Reinforcement Learning

Abstract

While real-world applications of reinforcement learning are becoming popular, the security and robustness of RL systems are worthy of more attention and exploration. In particular, recent works have revealed that, in a multi-agent RL environment, backdoor trigger actions can be injected into a victim agent (a.k.a. Trojan agent), which can result in a catastrophic failure as soon as it sees the backdoor trigger action. To ensure the security of RL agents against malicious backdoors, in this work, we propose the problem of Backdoor Detection in a multi-agent competitive reinforcement learning system, with the objective of detecting Trojan agents as well as the corresponding potential trigger actions, and further trying to mitigate their Trojan behavior. In order to solve this problem, we propose PolicyCleanse that is based on the property that the activated Trojan agents accumulated rewards degrade noticeably after several timesteps. Along with PolicyCleanse, we also design a machine unl

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyguo2022policycleanse

Related papers

• Backdoor Attacks On Multiagent Collaborative Systems (2022)0.00
• Provable Defense Against Backdoor Policies In Reinforcement Learning (2022)2.26
• Cooperative Backdoor Attack In Decentralized Reinforcement Learning With Theoretical Guarantee (2024)0.00
• Recover Triggered States: Protect Model Against Backdoor Attack In Reinforcement Learning (2023)0.00
• Trojdrl: Trojan Attacks On Deep Reinforcement Learning Agents (2019)0.00
• Adversarial Policies: Attacking Deep Reinforcement Learning (2019)0.00
• Neutral Agent-based Adversarial Policy Learning Against Deep Reinforcement Learning In Multi-party Open Systems (2025)0.00
• Toward Evaluating Robustness Of Reinforcement Learning With Adversarial Policy (2023)4.52