Bayesian Policy Gradients Via Alpha Divergence Dropout Inference

Abstract

Policy gradient methods have had great success in solving continuous control tasks, yet the stochastic nature of such problems makes deterministic value estimation difficult. We propose an approach which instead estimates a distribution by fitting the value function with a Bayesian Neural Network. We optimize an \(\alpha\)-divergence objective with Bayesian dropout approximation to learn and estimate this distribution. We show that using the Monte Carlo posterior mean of the Bayesian value function distribution, rather than a deterministic network, improves stability and performance of policy gradient methods in continuous control MuJoCo simulations.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyhenderson2017bayesian

Related papers

• Policy Search By Target Distribution Learning For Continuous Control (2019)3.58
• Bayesian Distributional Policy Gradients (2021)0.00
• Deep Bayesian Quadrature Policy Optimization (2020)0.00
• Learning Optimal Deterministic Policies With Stochastic Policy Gradients (2024)0.00
• Distributional Policy Optimization: An Alternative Approach For Continuous Control (2019)0.00
• Off-policy Policy Gradient With State Distribution Correction (2019)0.00
• An Empirical Analysis Of Measure-valued Derivatives For Policy Gradients (2021)0.00
• On The Model-based Stochastic Value Gradient For Continuous Reinforcement Learning (2020)0.00