Fully Parameterized Quantile Function For Distributional Reinforcement Learning

Abstract

Distributional Reinforcement Learning (RL) differs from traditional RL in that, rather than the expectation of total returns, it estimates distributions and has achieved state-of-the-art performance on Atari Games. The key challenge in practical distributional RL algorithms lies in how to parameterize estimated distributions so as to better approximate the true continuous distribution. Existing distributional RL algorithms parameterize either the probability side or the return value side of the distribution function, leaving the other side uniformly fixed as in C51, QR-DQN or randomly sampled as in IQN. In this paper, we propose fully parameterized quantile function that parameterizes both the quantile fraction axis (i.e., the x-axis) and the value axis (i.e., y-axis) for distributional RL. Our algorithm contains a fraction proposal network that generates a discrete set of quantile fractions and a quantile value network that gives corresponding quantile values. The two networks are joi

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyyang2019fully

Related papers

• Distributional Reinforcement Learning With Quantile Regression (2017)19.20
• Non-decreasing Quantile Function Network With Efficient Exploration For Distributional Reinforcement Learning (2021)4.52
• A Robust Quantile Huber Loss With Interpretable Parameter Adjustment In Distributional Reinforcement Learning (2024)0.00
• Distributional Reinforcement Learning With Dual Expectile-quantile Regression (2023)0.00
• Distributional Reinforcement Learning For Multi-dimensional Reward Functions (2021)0.00
• A Differential Perspective On Distributional Reinforcement Learning (2025)0.00
• Invariance To Quantile Selection In Distributional Continuous Control (2022)0.00
• Off-policy Distributional Q(\(\lambda\)): Distributional RL Without Importance Sampling (2024)0.00