Normality-guided Distributional Reinforcement Learning For Continuous Control

Abstract

Learning a predictive model of the mean return, or value function, plays a critical role in many reinforcement learning algorithms. Distributional reinforcement learning (DRL) has been shown to improve performance by modeling the value distribution, not just the mean. We study the value distribution in several continuous control tasks and find that the learned value distribution is empirically quite close to normal. We design a method that exploits this property, employing variances predicted from a variance network, along with returns, to analytically compute target quantile bars representing a normal for our distributional value function. In addition, we propose a policy update strategy based on the correctness as measured by structural characteristics of the value distribution not present in the standard value function. The approach we outline is compatible with many DRL structures. We use two representative on-policy algorithms, PPO and TRPO, as testbeds. Our method yields statisti

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keybyun2022normality

Related papers

• A Distributional Perspective On Reinforcement Learning (2017)0.00
• Continuous Control Reinforcement Learning: Distributed Distributional Drq Algorithms (2024)0.00
• A Differential Perspective On Distributional Reinforcement Learning (2025)0.00
• Distributional Reinforcement Learning For Multi-dimensional Reward Functions (2021)0.00
• Value-distributional Model-based Reinforcement Learning (2023)1.56
• Distributional Reinforcement Learning With Quantile Regression (2017)19.20
• Moments Matter:stabilizing Policy Optimization Using Return Distributions (2026)0.00
• A Comparative Analysis Of Expected And Distributional Reinforcement Learning (2019)9.76