Bayesian Policy Optimization For Model Uncertainty

Abstract

Addressing uncertainty is critical for autonomous systems to robustly adapt to the real world. We formulate the problem of model uncertainty as a continuous Bayes-Adaptive Markov Decision Process (BAMDP), where an agent maintains a posterior distribution over latent model parameters given a history of observations and maximizes its expected long-term reward with respect to this belief distribution. Our algorithm, Bayesian Policy Optimization, builds on recent policy optimization algorithms to learn a universal policy that navigates the exploration-exploitation trade-off to maximize the Bayesian value function. To address challenges from discretizing the continuous latent parameter space, we propose a new policy network architecture that encodes the belief distribution independently from the observable state. Our method significantly outperforms algorithms that address model uncertainty without explicitly reasoning about belief distributions and is competitive with state-of-the-art Part

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keylee2018bayesian

Related papers

• Deep Model-based Reinforcement Learning Via Estimated Uncertainty And Conservative Policy Optimization (2019)0.00
• Bayes-adaptive Deep Model-based Policy Optimisation (2020)0.00
• Bayes-cpace: PAC Optimal Exploration In Continuous Space Bayes-adaptive Markov Decision Processes (2018)0.00
• Sequential Monte Carlo For Policy Optimization In Continuous Pomdps (2025)0.00
• Uncertainty-aware Policy Optimization: A Robust, Adaptive Trust Region Approach (2020)0.00
• Offline Bayesian Aleatoric And Epistemic Uncertainty Quantification And Posterior Value Optimisation In Finite-state Mdps (2024)0.95
• Model-based Epistemic Variance Of Values For Risk-aware Policy Optimization (2023)0.00
• Bayesian Risk-sensitive Policy Optimization For Mdps With General Loss Functions (2025)0.00