Vizdoom: DRQN With Prioritized Experience Replay, Double-q Learning, & Snapshot Ensembling

Abstract

ViZDoom is a robust, first-person shooter reinforcement learning environment, characterized by a significant degree of latent state information. In this paper, double-Q learning and prioritized experience replay methods are tested under a certain ViZDoom combat scenario using a competitive deep recurrent Q-network (DRQN) architecture. In addition, an ensembling technique known as snapshot ensembling is employed using a specific annealed learning rate to observe differences in ensembling efficacy under these two methods. Annealed learning rates are important in general to the training of deep neural network models, as they shake up the status-quo and counter a model's tending towards local optima. While both variants show performance exceeding those of built-in AI agents of the game, the known stabilizing effects of double-Q learning are illustrated, and priority experience replay is again validated in its usefulness by showing immediate results early on in agent development, with the c

Stats

• citations13
• S2 citations—
• github stars0
• HF likes0
• heat score8.60
• arxiv keyschulze2018vizdoom

Related papers

• Vizdoom Competitions: Playing Doom From Pixels (2018)13.79
• Agents That Listen: High-throughput Reinforcement Learning With Multiple Sensory Systems (2021)8.09
• Deep Reinforcement Learning For Doom Using Unsupervised Auxiliary Tasks (2018)0.00
• Autoencoder-augmented Neuroevolution For Visual Doom Playing (2017)11.49
• Deep Reinforcement Learning With Quantum-inspired Experience Replay (2021)0.00
• Language Is Power: Representing States Using Natural Language In Reinforcement Learning (2019)0.00
• Continual Reinforcement Learning In 3D Non-stationary Environments (2019)0.00
• Douzero: Mastering Doudizhu With Self-play Deep Reinforcement Learning (2021)0.00