Application Of Linear Regression And Quasi-newton Methods To The Deep Reinforcement Learning In Continuous Action Cases

Abstract

The linear regression (LR) method offers the advantage that optimal parameters can be calculated relatively easily, although its representation capability is limited than that of the deep learning technique. To improve deep reinforcement learning, the Least Squares Deep Q Network (LS-DQN) method was proposed by Levine et al., which combines Deep Q Network (DQN) with LR method. However, the LS-DQN method assumes that the actions are discrete. In this study, we propose the Double Least Squares Deep Deterministic Policy Gradient (DLS-DDPG) method to address this limitation. This method combines the LR method with the Deep Deterministic Policy Gradient (DDPG) technique, one of the representative deep reinforcement learning algorithms for continuous action cases. For the LR update of the critic network, DLS-DDPG uses an algorithm similar to the Fitted Q iteration, the method which LS-DQN adopted. In addition, we calculated the optimal action using the quasi-Newton method and used it as both

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keykomatsu2025application

Related papers

• Deep Multi-agent Reinforcement Learning With Discrete-continuous Hybrid Action Spaces (2019)12.47
• Broad Critic Deep Actor Reinforcement Learning For Continuous Control (2024)0.00
• Deterministic Policy Gradient For Reinforcement Learning With Continuous Time And State (2025)0.00
• Sublinear Regret For A Class Of Continuous-time Linear-quadratic Reinforcement Learning Problems (2024)0.00
• Recursive Least Squares Advantage Actor-critic Algorithms (2022)0.00
• Performing Deep Recurrent Double Q-learning For Atari Games (2019)10.07
• On Improving Deep Reinforcement Learning For Pomdps (2017)0.00
• Comparing Deep Reinforcement Learning And Evolutionary Methods In Continuous Control (2017)0.00