Stlgame: Signal Temporal Logic Games In Adversarial Multi-agent Systems

Abstract

We study how to synthesize a robust and safe policy for autonomous systems under signal temporal logic (STL) tasks in adversarial settings against unknown dynamic agents. To ensure the worst-case STL satisfaction, we propose STLGame, a framework that models the multi-agent system as a two-player zero-sum game, where the ego agents try to maximize the STL satisfaction and other agents minimize it. STLGame aims to find a Nash equilibrium policy profile, which is the best case in terms of robustness against unseen opponent policies, by using the fictitious self-play (FSP) framework. FSP iteratively converges to a Nash profile, even in games set in continuous state-action spaces. We propose a gradient-based method with differentiable STL formulas, which is crucial in continuous settings to approximate the best responses at each iteration of FSP. We show this key aspect experimentally by comparing with reinforcement learning-based methods to find the best response. Experiments on two standa

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyyang2024stlgame

Related papers

• Funnel-based Reward Shaping For Signal Temporal Logic Tasks In Reinforcement Learning (2022)7.16
• TGPO: Temporal Grounded Policy Optimization For Signal Temporal Logic Tasks (2025)0.00
• Stackelberg Games For Learning Emergent Behaviors During Competitive Autocurricula (2023)5.84
• Decentralized Multi-agent Reinforcement Learning For Continuous-space Stochastic Games (2023)5.24
• Joint Learning Of Policy With Unknown Temporal Constraints For Safe Reinforcement Learning (2023)0.00
• Stratifying Reinforcement Learning With Signal Temporal Logic (2026)0.00
• Multi-agent Transformer-accelerated RL For Satisfaction Of STL Specifications (2024)0.00
• Independent Learning In Stochastic Games (2021)6.77