Efficient Reinforcement Learning For Starcraft By Abstract Forward Models And Transfer Learning

Abstract

Injecting human knowledge is an effective way to accelerate reinforcement learning (RL). However, these methods are underexplored. This paper presents our discovery that an abstract forward model (thought-game (TG)) combined with transfer learning (TL) is an effective way. We take StarCraft II as our study environment. With the help of a designed TG, the agent can learn a 99% win-rate on a 64x64 map against the Level-7 built-in AI, using only 1.08 hours in a single commercial machine. We also show that the TG method is not as restrictive as it was thought to be. It can work with roughly designed TGs, and can also be useful when the environment changes. Comparing with previous model-based RL, we show TG is more effective. We also present a TG hypothesis that gives the influence of different fidelity levels of TG. For real games that have unequal state and action spaces, we proposed a novel XfrNet of which usefulness is validated while achieving a 90% win-rate against the cheating Level-

Stats

• citations13
• S2 citations—
• github stars0
• HF likes0
• heat score8.60
• arxiv keyliu2019efficient

Related papers

• Applying Supervised And Reinforcement Learning Methods To Create Neural-network-based Agents For Playing Starcraft II (2021)0.00
• Starcraft Micromanagement With Reinforcement Learning And Curriculum Transfer Learning (2018)16.19
• SCC: An Efficient Deep Reinforcement Learning Agent Mastering The Game Of Starcraft II (2020)0.00
• Hierarchical Reinforcement Learning In Starcraft II With Human Expertise In Subgoals Selection (2020)0.00
• Tstarbot-x: An Open-sourced And Comprehensive Study For Efficient League Training In Starcraft II Full Game (2020)0.00
• Macro Action Selection With Deep Reinforcement Learning In Starcraft (2018)9.92
• Starcraft II: A New Challenge For Reinforcement Learning (2017)0.00
• High-level Strategy Selection Under Partial Observability In Starcraft: Brood War (2018)0.00