Uncertainty-aware Policy Optimization: A Robust, Adaptive Trust Region Approach

Abstract

In order for reinforcement learning techniques to be useful in real-world decision making processes, they must be able to produce robust performance from limited data. Deep policy optimization methods have achieved impressive results on complex tasks, but their real-world adoption remains limited because they often require significant amounts of data to succeed. When combined with small sample sizes, these methods can result in unstable learning due to their reliance on high-dimensional sample-based estimates. In this work, we develop techniques to control the uncertainty introduced by these estimates. We leverage these techniques to propose a deep policy optimization approach designed to produce stable performance even when data is scarce. The resulting algorithm, Uncertainty-Aware Trust Region Policy Optimization, generates robust policy updates that adapt to the level of uncertainty present throughout the learning process.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyqueeney2020uncertainty

Related papers

• Value Enhancement Of Reinforcement Learning Via Efficient And Robust Trust Region Optimization (2023)0.00
• Deep Model-based Reinforcement Learning Via Estimated Uncertainty And Conservative Policy Optimization (2019)0.00
• Adaptive Trust Region Policy Optimization: Global Convergence And Faster Rates For Regularized Mdps (2019)12.10
• Robust Adversarial Policy Optimization Under Dynamics Uncertainty (2026)0.00
• How To Enable Uncertainty Estimation In Proximal Policy Optimization (2022)0.00
• Bayesian Policy Optimization For Model Uncertainty (2018)0.00
• Simple Policy Optimization (2024)0.00
• Average-reward Reinforcement Learning With Trust Region Methods (2021)0.00