Continuous-action Reinforcement Learning For Playing Racing Games: Comparing SPG To PPO

Abstract

In this paper, a novel racing environment for OpenAI Gym is introduced. This environment operates with continuous action- and state-spaces and requires agents to learn to control the acceleration and steering of a car while navigating a randomly generated racetrack. Different versions of two actor-critic learning algorithms are tested on this environment: Sampled Policy Gradient (SPG) and Proximal Policy Optimization (PPO). An extension of SPG is introduced that aims to improve learning performance by weighting action samples during the policy update step. The effect of using experience replay (ER) is also investigated. To this end, a modification to PPO is introduced that allows for training using old action samples by optimizing the actor in log space. Finally, a new technique for performing ER is tested that aims to improve learning speed without sacrificing performance by splitting the training into two parts, whereby networks are first trained using state transitions from the repl

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyholubar2020continuous

Related papers

• Proximal Policy Optimization With Continuous Bounded Action Space Via The Beta Distribution (2021)0.00
• Proximal Policy Optimization Algorithms (2017)0.00
• Rewarded Region Replay (R3) For Policy Learning With Discrete Action Space (2024)0.95
• The Surprising Effectiveness Of PPO In Cooperative, Multi-agent Games (2021)0.00
• Unified Policy Optimization For Continuous-action Reinforcement Learning In Non-stationary Tasks And Games (2022)2.26
• Proximal Policy Optimization Via Enhanced Exploration Efficiency (2020)13.70
• PTR-PPO: Proximal Policy Optimization With Prioritized Trajectory Replay (2021)0.00
• Revisiting Design Choices In Proximal Policy Optimization (2020)0.00