Conjugated Discrete Distributions For Distributional Reinforcement Learning

Abstract

In this work we continue to build upon recent advances in reinforcement learning for finite Markov processes. A common approach among previous existing algorithms, both single-actor and distributed, is to either clip rewards or to apply a transformation method on Q-functions to handle a large variety of magnitudes in real discounted returns. We theoretically show that one of the most successful methods may not yield an optimal policy if we have a non-deterministic process. As a solution, we argue that distributional reinforcement learning lends itself to remedy this situation completely. By the introduction of a conjugated distributional operator we may handle a large class of transformations for real returns with guaranteed theoretical convergence. We propose an approximating single-actor algorithm based on this operator that trains agents directly on unaltered rewards using a proper distributional metric given by the Cram\'er distance. To evaluate its performance in a stochastic sett

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keylindenberg2021conjugated

Related papers

• Distributional Reinforcement Learning With Linear Function Approximation (2019)0.00
• Distributional Method For Risk Averse Reinforcement Learning (2023)0.00
• A Differential Perspective On Distributional Reinforcement Learning (2025)0.00
• Tractable Representations For Convergent Approximation Of Distributional HJB Equations (2025)0.00
• A Distributional Analysis Of Sampling-based Reinforcement Learning Algorithms (2020)0.00
• Distributional Reinforcement Learning For Multi-dimensional Reward Functions (2021)0.00
• Continuous Control Reinforcement Learning: Distributed Distributional Drq Algorithms (2024)0.00
• Distributional Soft Actor-critic With Diffusion Policy (2025)0.00