BAFFLE: Hiding Backdoors In Offline Reinforcement Learning Datasets

Abstract

Reinforcement learning (RL) makes an agent learn from trial-and-error experiences gathered during the interaction with the environment. Recently, offline RL has become a popular RL paradigm because it saves the interactions with environments. In offline RL, data providers share large pre-collected datasets, and others can train high-quality agents without interacting with the environments. This paradigm has demonstrated effectiveness in critical tasks like robot control, autonomous driving, etc. However, less attention is paid to investigating the security threats to the offline RL system. This paper focuses on backdoor attacks, where some perturbations are added to the data (observations) such that given normal observations, the agent takes high-rewards actions, and low-reward actions on observations injected with triggers. In this paper, we propose Baffle (Backdoor Attack for Offline Reinforcement Learning), an approach that automatically implants backdoors to RL agents by poisoning

Stats

• citations6
• S2 citations—
• github stars0
• HF likes0
• heat score6.34
• arxiv keygong2022baffle

Related papers

• Towards Robust Policy: Enhancing Offline Reinforcement Learning With Adversarial Attacks And Defenses (2024)3.58
• Adversarial Inception Backdoor Attacks Against Reinforcement Learning (2024)0.00
• AWAC: Accelerating Online Reinforcement Learning With Offline Datasets (2020)0.00
• Beyond Training-time Poisoning: Component-level And Post-training Backdoors In Deep Reinforcement Learning (2025)0.00
• Beware Untrusted Simulators -- Reward-free Backdoor Attacks In Reinforcement Learning (2026)0.00
• D4RL: Datasets For Deep Data-driven Reinforcement Learning (2020)0.00
• Cooperative Backdoor Attack In Decentralized Reinforcement Learning With Theoretical Guarantee (2024)0.00
• Sleepernets: Universal Backdoor Poisoning Attacks Against Reinforcement Learning Agents (2024)0.00