Distributional Reinforcement Learning For Multi-dimensional Reward Functions

Abstract

A growing trend for value-based reinforcement learning (RL) algorithms is to capture more information than scalar value functions in the value network. One of the most well-known methods in this branch is distributional RL, which models return distribution instead of scalar value. In another line of work, hybrid reward architectures (HRA) in RL have studied to model source-specific value functions for each source of reward, which is also shown to be beneficial in performance. To fully inherit the benefits of distributional RL and hybrid reward architectures, we introduce Multi-Dimensional Distributional DQN (MD3QN), which extends distributional RL to model the joint return distribution from multiple reward sources. As a by-product of joint distribution modeling, MD3QN can capture not only the randomness in returns for each source of reward, but also the rich reward correlation between the randomness of different sources. We prove the convergence for the joint distributional Bellman ope

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyzhang2021distributional

Related papers

• Distributional Reward Estimation For Effective Multi-agent Deep Reinforcement Learning (2022)0.00
• A Differential Perspective On Distributional Reinforcement Learning (2025)0.00
• A Distributional Perspective On Reinforcement Learning (2017)0.00
• Non-decreasing Quantile Function Network With Efficient Exploration For Distributional Reinforcement Learning (2021)4.52
• Conjugated Discrete Distributions For Distributional Reinforcement Learning (2021)0.00
• Normality-guided Distributional Reinforcement Learning For Continuous Control (2022)0.00
• Distributional Reinforcement Learning With Quantile Regression (2017)19.20
• Fully Parameterized Quantile Function For Distributional Reinforcement Learning (2019)0.00