A Robust Quantile Huber Loss With Interpretable Parameter Adjustment In Distributional Reinforcement Learning
2024 · Parvin Malekzadeh, Konstantinos N. Plataniotis, Zissis Poulos, et al.
Abstract
Distributional Reinforcement Learning (RL) estimates return distribution mainly by learning quantile values via minimizing the quantile Huber loss function, entailing a threshold parameter often selected heuristically or via hyperparameter search, which may not generalize well and can be suboptimal. This paper introduces a generalized quantile Huber loss function derived from Wasserstein distance (WD) calculation between Gaussian distributions, capturing noise in predicted (current) and target (Bellman-updated) quantile values. Compared to the classical quantile Huber loss, this innovative loss function enhances robustness against outliers. Notably, the classical Huber loss function can be seen as an approximation of our proposed loss, enabling parameter adjustment by approximating the amount of noise in the data during the learning process. Empirical tests on Atari games, a common application in distributional RL, and a recent hedging strategy using distributional RL, validate the eff
Authors
(none)
Tags
Stats
Related papers
- Fully Parameterized Quantile Function For Distributional Reinforcement Learning (2019)0.00
- Distributional Reinforcement Learning With Quantile Regression (2017)19.20
- Distributional Reinforcement Learning With Dual Expectile-quantile Regression (2023)0.00
- Non-decreasing Quantile Function Network With Efficient Exploration For Distributional Reinforcement Learning (2021)4.52
- Distributional Reinforcement Learning With Regularized Wasserstein Loss (2022)0.00
- Exploring The Training Robustness Of Distributional Reinforcement Learning Against Noisy State Observations (2021)0.00
- Intrinsic Benefits Of Categorical Distributional Loss: Uncertainty-aware Regularized Exploration In Reinforcement Learning (2021)0.00
- A Differential Perspective On Distributional Reinforcement Learning (2025)0.00