Inverse-inverse Reinforcement Learning. How To Hide Strategy From An Adversarial Inverse Reinforcement Learner

Abstract

Inverse reinforcement learning (IRL) deals with estimating an agent's utility function from its actions. In this paper, we consider how an agent can hide its strategy and mitigate an adversarial IRL attack; we call this inverse IRL (I-IRL). How should the decision maker choose its response to ensure a poor reconstruction of its strategy by an adversary performing IRL to estimate the agent's strategy? This paper comprises four results: First, we present an adversarial IRL algorithm that estimates the agent's strategy while controlling the agent's utility function. Our second result for I-IRL result spoofs the IRL algorithm used by the adversary. Our I-IRL results are based on revealed preference theory in micro-economics. The key idea is for the agent to deliberately choose sub-optimal responses that sufficiently masks its true strategy. Third, we give a sample complexity result for our main I-IRL result when the agent has noisy estimates of the adversary specified utility function. Fin

Stats

• citations9
• S2 citations—
• github stars0
• HF likes0
• heat score7.50
• arxiv keypattanayak2022inverse

Related papers

• Towards Theoretical Understanding Of Inverse Reinforcement Learning (2023)0.00
• Inverse Reinforcement Learning Using Revealed Preferences And Passive Stochastic Optimization (2025)0.00
• Inverse Reinforcement Learning Without Reinforcement Learning (2023)0.00
• Offline Inverse RL: New Solution Concepts And Provably Efficient Algorithms (2024)0.00
• Non-cooperative Inverse Reinforcement Learning (2019)0.00
• Partial Identifiability And Misspecification In Inverse Reinforcement Learning (2024)0.00
• Active Learning For Risk-sensitive Inverse Reinforcement Learning (2019)0.00
• Is Inverse Reinforcement Learning Harder Than Standard Reinforcement Learning? A Theoretical Perspective (2023)0.00