Investigating The Impact Of Action Representations In Policy Gradient Algorithms

Abstract

Reinforcement learning~(RL) is a versatile framework for learning to solve complex real-world tasks. However, influences on the learning performance of RL algorithms are often poorly understood in practice. We discuss different analysis techniques and assess their effectiveness for investigating the impact of action representations in RL. Our experiments demonstrate that the action representation can significantly influence the learning performance on popular RL benchmark tasks. The analysis results indicate that some of the performance differences can be attributed to changes in the complexity of the optimization landscape. Finally, we discuss open challenges of analysis techniques for RL algorithms.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyschneider2023investigating

Related papers

• Improving Policy Gradient By Exploring Under-appreciated Rewards (2016)0.00
• Policy Gradient RL Algorithms As Directed Acyclic Graphs (2020)0.00
• What Matters In On-policy Reinforcement Learning? A Large-scale Empirical Study (2020)0.00
• Discovering Reinforcement Learning Algorithms (2020)0.00
• Where Did My Optimum Go?: An Empirical Analysis Of Gradient Descent Optimization In Policy Gradient Methods (2018)0.00
• S-REINFORCE: A Neuro-symbolic Policy Gradient Approach For Interpretable Reinforcement Learning (2023)0.00
• Representation-driven Reinforcement Learning (2023)0.00
• Learning Optimal Deterministic Policies With Stochastic Policy Gradients (2024)0.00