Sampling Attacks On Meta Reinforcement Learning: A Minimax Formulation And Complexity Analysis

Abstract

Meta reinforcement learning (meta RL), as a combination of meta-learning ideas and reinforcement learning (RL), enables the agent to adapt to different tasks using a few samples. However, this sampling-based adaptation also makes meta RL vulnerable to adversarial attacks. By manipulating the reward feedback from sampling processes in meta RL, an attacker can mislead the agent into building wrong knowledge from training experience, which deteriorates the agent's performance when dealing with different tasks after adaptation. This paper provides a game-theoretical underpinning for understanding this type of security risk. In particular, we formally define the sampling attack model as a Stackelberg game between the attacker and the agent, which yields a minimax formulation. It leads to two online attack schemes: Intermittent Attack and Persistent Attack, which enable the attacker to learn an optimal sampling attack, defined by an \(\epsilon\)-first-order stationary point, within \(\mathca

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyli2022sampling

Related papers

• Optimal Attack And Defense For Reinforcement Learning (2023)6.34
• Model-based Adversarial Meta-reinforcement Learning (2020)0.00
• Learning To Cope With Adversarial Attacks (2019)0.00
• Memory Sequence Length Of Data Sampling Impacts The Adaptation Of Meta-reinforcement Learning Agents (2024)2.26
• Understanding The Limits Of Poisoning Attacks In Episodic Reinforcement Learning (2022)3.58
• Harnessing The Power Of Reinforcement Learning For Adaptive MCMC (2025)0.00
• First-explore, Then Exploit: Meta-learning To Solve Hard Exploration-exploitation Trade-offs (2023)0.00
• Targeted Adversarial Attacks On Deep Reinforcement Learning Policies Via Model Checking (2022)2.26