DNS: Determinantal Point Process Based Neural Network Sampler For Ensemble Reinforcement Learning

Abstract

Application of ensemble of neural networks is becoming an imminent tool for advancing the state-of-the-art in deep reinforcement learning algorithms. However, training these large numbers of neural networks in the ensemble has an exceedingly high computation cost which may become a hindrance in training large-scale systems. In this paper, we propose DNS: a Determinantal Point Process based Neural Network Sampler that specifically uses k-dpp to sample a subset of neural networks for backpropagation at every training step thus significantly reducing the training time and computation cost. We integrated DNS in REDQ for continuous control tasks and evaluated on MuJoCo environments. Our experiments show that DNS augmented REDQ outperforms baseline REDQ in terms of average cumulative reward and achieves this using less than 50% computation when measured in FLOPS.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keysheikh2022dns

Related papers

• Accelerating Distributed Deep Reinforcement Learning By In-network Experience Sampling (2021)2.26
• Multi-agent Determinantal Q-learning (2020)0.00
• Dropout Q-functions For Doubly Efficient Reinforcement Learning (2021)0.00
• Rl-pinns: Reinforcement Learning-driven Adaptive Sampling For Efficient Training Of Pinns (2025)0.00
• Graying The Black Box: Understanding Dqns (2016)0.00
• Ardns-fn-quantum: A Quantum-enhanced Reinforcement Learning Framework With Cognitive-inspired Adaptive Exploration For Dynamic Environments (2025)2.26
• NROWAN-DQN: A Stable Noisy Network With Noise Reduction And Online Weight Adjustment For Exploration (2020)0.00
• Aggressive Q-learning With Ensembles: Achieving Both High Sample Efficiency And High Asymptotic Performance (2021)0.00