Episodic Exploration For Deep Deterministic Policies: An Application To Starcraft Micromanagement Tasks

Abstract

We consider scenarios from the real-time strategy game StarCraft as new benchmarks for reinforcement learning algorithms. We propose micromanagement tasks, which present the problem of the short-term, low-level control of army members during a battle. From a reinforcement learning point of view, these scenarios are challenging because the state-action space is very large, and because there is no obvious feature representation for the state-action evaluation function. We describe our approach to tackle the micromanagement scenarios with deep neural network controllers from raw state features given by the game engine. In addition, we present a heuristic reinforcement learning algorithm which combines direct exploration in the policy space and backpropagation. This algorithm allows for the collection of traces for learning using deterministic policies, which appears much more efficient than, for example, \{\epsilon\}-greedy exploration. Experiments show that with this algorithm, we succes

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyusunier2016episodic

Related papers

• Starcraft Micromanagement With Reinforcement Learning And Curriculum Transfer Learning (2018)16.19
• Macro Action Selection With Deep Reinforcement Learning In Starcraft (2018)9.92
• Growing Action Spaces (2019)0.00
• High-level Strategy Selection Under Partial Observability In Starcraft: Brood War (2018)0.00
• Never Give Up: Learning Directed Exploration Strategies (2020)0.00
• Starcraft II: A New Challenge For Reinforcement Learning (2017)0.00
• Hierarchical Reinforcement Learning In Starcraft II With Human Expertise In Subgoals Selection (2020)0.00
• Applying Supervised And Reinforcement Learning Methods To Create Neural-network-based Agents For Playing Starcraft II (2021)0.00