Choosing Well Your Opponents: How To Guide The Synthesis Of Programmatic Strategies

Abstract

This paper introduces Local Learner (2L), an algorithm for providing a set of reference strategies to guide the search for programmatic strategies in two-player zero-sum games. Previous learning algorithms, such as Iterated Best Response (IBR), Fictitious Play (FP), and Double-Oracle (DO), can be computationally expensive or miss important information for guiding search algorithms. 2L actively selects a set of reference strategies to improve the search signal. We empirically demonstrate the advantages of our approach while guiding a local search algorithm for synthesizing strategies in three games, including MicroRTS, a challenging real-time strategy game. Results show that 2L learns reference strategies that provide a stronger search signal than IBR, FP, and DO. We also simulate a tournament of MicroRTS, where a synthesizer using 2L outperformed the winners of the two latest MicroRTS competitions, which were programmatic strategies written by human programmers.

Stats

• citations2
• S2 citations—
• github stars0
• HF likes0
• heat score3.58
• arxiv keymoraes2023choosing

Related papers

• Evaluation And Learning In Two-player Symmetric Games Via Best And Better Responses (2022)0.00
• High-level Strategy Selection Under Partial Observability In Starcraft: Brood War (2018)0.00
• A Deep Reinforcement Learning Approach For Finding Non-exploitable Strategies In Two-player Atari Games (2022)0.00
• Reinforcing Competitive Multi-agents For Playing 'so Long Sucker' (2024)0.00
• All By Myself: Learning Individualized Competitive Behaviour With A Contrastive Reinforcement Learning Optimization (2023)7.16
• Strategy Synthesis In Markov Decision Processes Under Limited Sampling Access (2023)0.00
• Approximate Exploitability: Learning A Best Response In Large Games (2020)0.00
• Neural Auto-curricula (2021)0.00