M\(^2\)DQN: A Robust Method For Accelerating Deep Q-learning Network

Abstract

Deep Q-learning Network (DQN) is a successful way which combines reinforcement learning with deep neural networks and leads to a widespread application of reinforcement learning. One challenging problem when applying DQN or other reinforcement learning algorithms to real world problem is data collection. Therefore, how to improve data efficiency is one of the most important problems in the research of reinforcement learning. In this paper, we propose a framework which uses the Max-Mean loss in Deep Q-Network (M\(^2\)DQN). Instead of sampling one batch of experiences in the training step, we sample several batches from the experience replay and update the parameters such that the maximum TD-error of these batches is minimized. The proposed method can be combined with most of existing techniques of DQN algorithm by replacing the loss function. We verify the effectiveness of this framework with one of the most widely used techniques, Double DQN (DDQN), in several gym games. The results sh

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyzhang2022m

Related papers

• Deep Q-networks For Accelerating The Training Of Deep Neural Networks (2016)0.00
• An Adaptive Synchronization Approach For Weights Of Deep Reinforcement Learning (2020)0.00
• A Theoretical Analysis Of Deep Q-learning (2019)0.00
• Convergent And Efficient Deep Q Network Algorithm (2021)0.00
• Elastic Step DQN: A Novel Multi-step Algorithm To Alleviate Overestimation In Deep Qnetworks (2022)10.85
• DQN With Model-based Exploration: Efficient Learning On Environments With Sparse Rewards (2019)0.00
• Weighted Double Deep Multiagent Reinforcement Learning In Stochastic Cooperative Environments (2018)0.00
• Deep Q-network Based Multi-agent Reinforcement Learning With Binary Action Agents (2020)0.00