Model-free Reinforcement Learning For Stochastic Stackelberg Security Games

Abstract

In this paper, we consider a sequential stochastic Stackelberg game with two players, a leader and a follower. The follower has access to the state of the system while the leader does not. Assuming that the players act in their respective best interests, the follower's strategy is to play the best response to the leader's strategy. In such a scenario, the leader has the advantage of committing to a policy which maximizes its own returns given the knowledge that the follower is going to play the best response to its policy. Thus, both players converge to a pair of policies that form the Stackelberg equilibrium of the game. Recently,~[1] provided a sequential decomposition algorithm to compute the Stackelberg equilibrium for such games which allow for the computation of Markovian equilibrium policies in linear time as opposed to double exponential, as before. In this paper, we extend the idea to an MDP whose dynamics are not known to the players, to propose an RL algorithm based on Expec

Stats

• citations4
• S2 citations—
• github stars0
• HF likes0
• heat score5.24
• arxiv keymishra2020model

Related papers

• Can Reinforcement Learning Find Stackelberg-nash Equilibria In General-sum Markov Games With Myopic Followers? (2021)0.00
• Actions Speak What You Want: Provably Sample-efficient Reinforcement Learning Of The Quantal Stackelberg Equilibrium From Strategic Feedbacks (2023)0.00
• Sample-efficient Learning Of Stackelberg Equilibria In General-sum Games (2021)0.00
• Oracles & Followers: Stackelberg Equilibria In Deep Multi-agent Reinforcement Learning (2022)0.00
• Multi-agent Reinforcement Learning In Bayesian Stackelberg Markov Games For Adaptive Moving Target Defense (2020)0.00
• Online Learning In Stackelberg Games With An Omniscient Follower (2023)0.00
• Stackelberg Batch Policy Learning (2023)0.00
• Stackelberg POMDP: A Reinforcement Learning Approach For Economic Design (2022)0.00